Related papers: Mutual information superadditivity and unitarity b…
Mutual information $I(X;Y)$ is a useful definition in information theory to estimate how much information the random variable $Y$ holds about the random variable $X$. One way to define the mutual information is by comparing the joint…
We derive a tight lower bound on equivocation (conditional entropy), or equivalently a tight upper bound on mutual information between a signal variable and channel outputs. The bound is in terms of the joint distribution of the signals and…
We use holographic methods to characterize the RG flow of quantum information in a Chern-Simons theory coupled to massive fermions. First, we use entanglement entropy and mutual information between strips to derive the dimension of the…
We derive information theoretic generalization bounds for supervised learning algorithms based on a new measure of leave-one-out conditional mutual information (loo-CMI). Contrary to other CMI bounds, which are black-box bounds that do not…
We consider a model for a Planck scale ultraviolet cutoff which is based on Shannon sampling. Shannon sampling originated in information theory, where it expresses the equivalence of continuous and discrete representations of information.…
Mutual information I in infinite sequences (and in their finite prefixes) is essential in theoretical analysis of many situations. Yet its right definition has been elusive for a long time. I address it by generalizing Kolmogorov Complexity…
We propose a new framework for reasoning about information in complex systems. Our foundation is based on a variational extension of Shannon's information theory that takes into account the modeling power and computational constraints of…
We show that known entropy bounds constrain the information carried off by radiation to null infinity. We consider distant, planar null hypersurfaces in asymptotically flat spacetime. Their focussing and area loss can be computed…
We consider the problem of recovering the community structure in the stochastic block model with two communities. We aim to describe the mutual information between the observed network and the actual community structure in the sparse…
The often-asked question whether space-time is discrete or continuous may not be the right question to ask: Mathematically, it is possible that space-time possesses the differentiability properties of manifolds as well as the ultraviolet…
The uncertainty principle is one of the characteristic properties of quantum theory based on incompatibility. Apart from the incompatible relation of quantum states, mutually exclusiveness is another remarkable phenomenon in the…
Using the algebraic approach to entanglement entropy, we study several dual pairs of lattice theories and show how the entropy is completely preserved across each duality. Our main result is that a maximal algebra of observables in a region…
A trade-off relation on our knowledge about two noncommuting observables of a qubit system in simultaneous measurement is formulated. The obtained inequality offers a quantitative information-theoretic representation of Bohr's principle of…
The entanglement entropy in a quantum field theory between two regions of space has been shown in simple cases to be proportional to the volume of the hypersurface separating the regions. We prove that this is true for a free scalar field…
We are interested in learning data-driven representations that can generalize well, even when trained on inherently biased data. In particular, we face the case where some attributes (bias) of the data, if learned by the model, can severely…
The classical information metric provides a unique notion of distance on the space of probability distributions with a well-defined operational interpretation: two distributions are far apart if they are readily distinguishable from one…
The information metric provides a unique notion of distance along any continuous family of physical theories, placing two theories further apart the more distinguishable they are. As such, the information metric is typically singular at…
Recently we conjectured that a certain set of universal topological quantities characterize topological order in any dimension. Those quantities can be extracted from the universal overlap of the ground state wave functions. For systems…
Feature selection methods are usually evaluated by wrapping specific classifiers and datasets in the evaluation process, resulting very often in unfair comparisons between methods. In this work, we develop a theoretical framework that…
One of the main notions of information theory is the notion of mutual information in two messages (two random variables in Shannon information theory or two binary strings in algorithmic information theory). The mutual information in $x$…