Related papers: On Information Links
We correct claims about lower bounds on mutual information (MI) between real-valued random variables made in A. Kraskov {\it et al.}, Phys. Rev. E {\bf 69}, 066138 (2004). We show that non-trivial lower bounds on MI in terms of linear…
We consider the problem of recovering the community structure in the stochastic block model with two communities. We aim to describe the mutual information between the observed network and the actual community structure in the sparse…
A simpler approach to the characterization of vanishing conditional mutual information is presented. Some remarks are given as well. More specifically, relating the conditional mutual information to a commutator is a very promising approach…
Negative probabilities emerged at intermediate steps in various attempts to predict the distributions of quantum interference. There is no consensus on their meaning yet. It has been suggested (Khrennikov, 1998) that negative probabilities…
This paper presents the computational methods of information cohomology applied to genetic expression in and in the companion paper and proposes its interpretations in terms of statistical physics and machine learning. In order to further…
In this paper a numerical method is presented, which finds a lower bound for the mutual information between a binary and an arbitrary finite random variable with joint distributions that have a variational distance not greater than a known…
The introduction of the partial information decomposition generated a flurry of proposals for defining an intersection information that quantifies how much of "the same information" two or more random variables specify about a target random…
A three-parameter discrete distribution is developed to describe the multiplicity distributions observed in total- and limited phase space volumes in different collision processes. The probability law is obtained by the Poisson transform of…
We formulate and analyze the compound information bottleneck programming. In this problem, a Markov chain $ \mathsf{X} \rightarrow \mathsf{Y} \rightarrow \mathsf{Z} $ is assumed with fixed marginal distributions $\mathsf{P}_{\mathsf{X}}$…
In regression models for categorical data a linear model is typically related to the response variables via a transformation of probabilities called the link function. We introduce an approach based on two link functions for binary data…
We develop an information-theoretic perspective on some questions in convex geometry, providing for instance a new equipartition property for log-concave probability measures, some Gaussian comparison results for log-concave measures, an…
We present a quantum information theory that allows for the consistent description of quantum entanglement. It parallels classical (Shannon) information theory but is based entirely on density matrices, rather than probability…
In this paper we establish lower bounds on information divergence from a distribution to certain important classes of distributions as Gaussian, exponential, Gamma, Poisson, geometric, and binomial. These lower bounds are tight and for…
We generalize a result by Carlen and Cordero-Erausquin on the equivalence between the Brascamp-Lieb inequality and the subadditivity of relative entropy by allowing for random transformations (a broadcast channel). This leads to a unified…
The minimum rate needed to accurately approximate a product distribution based on an unnormalized informational divergence is shown to be a mutual information. This result subsumes results of Wyner on common information and Han-Verd\'{u} on…
Of the various attempts to generalize information theory to multiple variables, the most widely utilized, interaction information, suffers from the problem that it is sometimes negative. Here we reconsider from first principles the general…
Recently, Brassard et. al. conjectured that the fact that the maximal possible correlations between two non-local parties are the quantum-mechanical ones is linked to a reasonable restriction on communication complexity. We provide further…
Reconstructing the structural connectivity between interacting units from observed activity is a challenge across many different disciplines. The fundamental first step is to establish whether or to what extent the interactions between the…
We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections, a problem relevant in compressed sensing, sparse superposition codes or code division multiple access just to cite few. There has…
A C_k-move is a local move that involves (k+1) strands of a link. A C_k-move is called a C_k^d-move if these (k+1) strands belong to mutually distinct components of a link. Since a C_k^d-move preserves all k-component sublinks of a link, we…