相关论文: Mutual information decay for factors of IID

In a recent paper by A. Backhausz, B. Gerencs\'er and V. Harangi, it was shown that factors of independent identically distributed random processes (IID) on trees obey certain geometry-driven inequalities. In particular, the mutual…

This paper is concerned with certain invariant random processes (called factors of IID) on infinite trees. Given such a process, one can assign entropies to different finite subgraphs of the tree. There are linear inequalities between these…

The mutual information between two jointly distributed random variables $X$ and $Y$ is a functional of the joint distribution $P_{XY},$ which is sometimes difficult to handle or estimate. A coarser description of the statistical behavior of…

We study factor of i.i.d. processes on the $d$-regular tree for $d \geq 3$. We show that if such a process is restricted to two distant connected subgraphs of the tree, then the two parts are basically uncorrelated. More precisely, any…

This paper is devoted to the mathematical study of some divergences based on the mutual information well-suited to categorical random vectors. These divergences are generalizations of the "entropy distance" and "information distance". Their…

We derive a general upper bound to mutual information in terms of the Fisher information. The bound may be further used to derive a lower bound for the Bayesian quadratic cost. These two provide alternatives to other inequalities in the…

Mutual information is used as a purely geometrical regularization of entanglement entropy applicable to any QFT. A coefficient in the mutual information between concentric circular entangling surfaces gives a precise universal prescription…

We develop and formalize a phase diagnostic based on the information-distance \(d_E = K_0/\sqrt{I}\) (mutual information \(I\)) for 1D quantum chains. Calibrating with the Euclidean benchmark \(I(r)\propto r^{-2}\mapsto d_E(r)\propto r\)…

In this paper we investigate stopping criteria for iterative decoding from a mutual information perspective. We introduce new iteration stopping rules based on an approximation of the mutual information between encoded bits and decoder soft…

There is a vast body of recent literature on the reliability of communication through noisy channels, the recovery of community structures in the stochastic block model, the limiting behavior of the free entropy in spin glasses and the…

We derive a tight lower bound on equivocation (conditional entropy), or equivalently a tight upper bound on mutual information between a signal variable and channel outputs. The bound is in terms of the joint distribution of the signals and…

Let $G$ be a large-girth $d$-regular graph and $\mu$ be a random process on the vertices of $G$ produced by a randomized local algorithm. We prove the upper bound $(k+1-2k/d)\Bigl(\frac{1}{\sqrt{d-1}}\Bigr)^k$ for the (absolute value of…

For sensory networks, we determine the rate with which they acquire information about the changing external conditions. Comparing this rate with the thermodynamic entropy production that quantifies the cost of maintaining the network, we…

We consider the transmission of a state from the root of a tree towards its leaves, assuming that each transmission occurs through a noisy channel. The states at the leaves are observed, while at deeper nodes we can compute the likelihood…

Rooted trees with probabilities are used to analyze properties of a variable length code. A bound is derived on the difference between the entropy rates of the code and a memoryless source. The bound is in terms of normalized informational…

We develop an information-theoretic approach to isoperimetric inequalities based on entropy dissipation under heat flow. By viewing diffusion as a noisy information channel, we measure how mutual information about set membership decays over…

The amount of information exchanged per unit of time between two nodes in a dynamical network or between two data sets is a powerful concept for analysing complex systems. This quantity, known as the mutual information rate (MIR), is…

Information theoretic measures (entropies, entropy rates, mutual information) are nowadays commonly used in statistical signal processing for real-world data analysis. The present work proposes the use of Auto Mutual Information (Mutual…

The generalization error of a learning algorithm refers to the discrepancy between the loss of a learning algorithm on training data and that on unseen testing data. Various information-theoretic bounds on the generalization error have been…

For random $d$-regular graphs on $N$ vertices with $1 \ll d \ll N^{2/3}$, we develop a $d^{-1/2}$ expansion of the local eigenvalue distribution about the Kesten-McKay law up to order $d^{-3}$. This result is valid up to the edge of the…