Related papers: Mutual Information from Modular Flow in General CF…
Modular flow is a symmetry of the algebra of observables associated to spacetime regions. Being closely related to entanglement, it has played a key role in recent connections between information theory, QFT and gravity. However, little is…
The ability to compress observational data and accurately estimate physical parameters relies heavily on informative summary statistics. In this paper, we introduce the use of mutual information (MI) as a means of evaluating the quality of…
Mutual information serves as an important measure of correlation between subsystem components. In the framework of quantum field theories (QFTs) they have better regulated UV behavior than entanglement entropy, and thus provide more direct…
We study the behavior of the mutual information (MI) in various quadratic fermionic chains, with and without pairing terms and both with short- and long-range hoppings. The models considered include the short-range limit and long-range…
Meta-learning optimizes an inductive bias---typically in the form of the hyperparameters of a base-learning algorithm---by observing data from a finite number of related tasks. This paper presents an information-theoretic bound on the…
We study the generic scaling properties of the mutual information between two disjoint intervals, in a class of one-dimensional quantum critical systems described by the c=1 bosonic field theory. A numerical analysis of a spin-chain model…
Deep models produce a number of features in each internal layer. A key problem in applications such as feature compression for remote inference is determining how important each feature is for the task(s) performed by the model. The problem…
We apply a new technique, the mutual information (MI) from information theory, to time-distance helioseismology, and demonstrate that it can successfully reproduce several classic results based on the widely used cross-covariance method. MI…
We investigate the behaviour of the mutual information $\mathcal{I}_{AB}$ between two "small" and wide separated spherical regions $A$ and $B$ in the $\mathcal{N}=4$ SYM gauge theory dual to Type IIB string theory in $AdS_5 \times S^5$. To…
Renormalization is an essential technique in field-theoretic descriptions of natural phenomena, where the absence of a UV-complete description yields an abundance of divergent quantities. While the renormalization prescription has been…
Mutual Information (MI) is often used for feature selection when developing classifier models. Estimating the MI for a subset of features is often intractable. We demonstrate, that under the assumptions of conditional independence, MI…
Relational data augmentation is a powerful technique for enhancing data analytics and improving machine learning models by incorporating columns from external datasets. However, it is challenging to efficiently discover relevant external…
To provide an efficient approach to characterize the input-output mutual information (MI) under additive white Gaussian noise (AWGN) channel, this short report fits the curves of exact MI under multilevel quadrature amplitude modulation…
Many recent methods for unsupervised or self-supervised representation learning train feature extractors by maximizing an estimate of the mutual information (MI) between different views of the data. This comes with several immediate…
We propose a fully data-driven approach to designing mutual information (MI) estimators. Since any MI estimator is a function of the observed sample from two random variables, we parameterize this function with a neural network (MIST) and…
Estimating mutual information (MI) is a fundamental task in data science and machine learning. Existing estimators mainly rely on either highly flexible models (e.g., neural networks), which require large amounts of data, or overly…
We introduce the Mutual Information Machine (MIM), a novel formulation of representation learning, using a joint distribution over the observations and latent state in an encoder/decoder framework. Our key principles are symmetry and mutual…
The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions $\Pi_{\mathcal{A}}$, on small enough…
We provide an information-theoretic framework for studying the generalization properties of machine learning algorithms. Our framework ties together existing approaches, including uniform convergence bounds and recent methods for adaptive…
Subtraction of aligned images is a means to assess changes in a wide variety of clinical applications. In this paper we explore the information theoretical origin of Mutual Information (MI), which is based on Shannon's entropy.However, the…