Related papers: Mutual Information of Generalized Free Fields
We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…
We study different aspects of quantum field theory at finite density using methods from quantum information theory. For simplicity we focus on massive Dirac fermions with nonzero chemical potential, and work in $1+1$ space-time dimensions.…
The study of R\'enyi mutual information (RMI) sheds light on the AdS/CFT correspondence beyond classical order. In this article, we study the R\'enyi mutual information between two intervals at large distance in two-dimensional holographic…
In the last decade, recent successes in deep clustering majorly involved the Mutual Information (MI) as an unsupervised objective for training neural networks with increasing regularisations. While the quality of the regularisations have…
We study two entanglement measures in a large family of isotropic many-body states including incompressible quantum Hall liquids and quantum critical systems: the logarithmic negativity (LN), and mutual information (MI). For pure states,…
This thesis uses a quantity that is defined and justified by information theory -- mutual information -- to examine models of condensed matter systems. More precisely, it studies models which are made up out of ferromagnetically interacting…
To provide an efficient approach to characterize the input-output mutual information (MI) under additive white Gaussian noise (AWGN) channel, this short report fits the curves of exact MI under multilevel quadrature amplitude modulation…
We analyze the geometry of a joint distribution over a set of discrete random variables. We briefly review Shannon's entropy, conditional entropy, mutual information and conditional mutual information. We review the entropic information…
We study the structure of divergences and universal terms of the entanglement and R\'enyi entropies for singular regions. First, we show that for $(3+1)$-dimensional free conformal field theories (CFTs), entangling regions emanating from…
Multimodal image alignment is the process of finding spatial correspondences between images formed by different imaging techniques or under different conditions, to facilitate heterogeneous data fusion and correlative analysis. The…
We propose a unified information-geometric framework that formalizes understanding in learning as a trade-off between informativeness and geometric simplicity. An encoder phi is evaluated by U(phi) = I(phi(X); Y) - beta * C(phi), where…
Reshef & Reshef recently published a paper in which they present a method called the Maximal Information Coefficient (MIC) that can detect all forms of statistical dependence between pairs of variables as sample size goes to infinity. While…
Random factor graphs provide a powerful framework for the study of inference problems such as decoding problems or the stochastic block model. Information-theoretically the key quantity of interest is the mutual information between the…
A new interpretation of the basic vector |0> of the free Fock space (FFS) and the FFS is proposed. The approximations to various equations with additional parameters, for n-point information (n-pi), are also considered in the case of…
We present a new family of information-theoretic generalization bounds within the framework of conditional mutual information (CMI). Most of our results are established based on the leave-$m$-out (L$m$O) cross-validation error, with $m$…
Gauging is a powerful operation on symmetries in quantum field theory (QFT), as it connects distinct theories and also reveals hidden structures in a given theory. We initiate a systematic investigation of gauging discrete generalized…
Mutual information is one of the essential building blocks of information theory. Yet, it is only finitely defined for distributions with fast decaying tails on a countable joint alphabet of two random elements. The unboundedness of mutual…
We propose a new information-theoretic bound on generalization error based on a combination of the error decomposition technique of Bu et al. and the conditional mutual information (CMI) construction of Steinke and Zakynthinou. In a…
Estimating the Generalization Error (GE) of Deep Neural Networks (DNNs) is an important task that often relies on availability of held-out data. The ability to better predict GE based on a single training set may yield overarching DNN…
We introduce a geometric scaling relation that characterizes the local scale behavior of correlations using the informational distance $d_E = K_0/\sqrt{I}$, where $I$ is the mutual information. We define a geometric conversion factor, $G…