Related papers: Ribbons from Independence Structure: Hypercontract…
We say that a measure of dependence between two random variables $X$ and $Y$, denoted as $\rho(X;Y)$, satisfies the data processing property if $\rho(X;Y)\geq \rho(X';Y')$ for every $X'\rightarrow X\rightarrow Y\rightarrow Y'$, and…
A measure of correlation is said to have the tensorization property if it is unchanged when computed for i.i.d.\ copies. More precisely, a measure of correlation between two random variables $(X, Y)$ denoted by $\rho(X, Y)$, has the…
We derive new characterisations of the matrix $\mathrm{\Phi}$-entropy functionals introduced in [Electron.~J.~Probab., 19(20): 1--30, 2014]. Notably, all known equivalent characterisations of the classical $\Phi$-entropies have their matrix…
We examine a class of deep learning models with a tractable method to compute information-theoretic quantities. Our contributions are three-fold: (i) We show how entropies and mutual informations can be derived from heuristic statistical…
Entropic independence is a structural property of measures that underlies modern proofs of functional inequalities, notably (modified) log-Sobolev inequalities, via ``annealing'' or local-to-global schemes. Existing sufficient criteria for…
In 2000, Evans et al. [Eva+00] proved the subadditivity of the mutual information in the broadcasting on tree model with binary vertex labels and symmetric channels. They raised the question of whether such subadditivity extends to loopy…
The mixing time of a Markov chain determines how fast the iterates of the Markov chain converge to the stationary distribution; however, it does not control the dependencies between samples along the Markov chain. In this paper, we study…
We present new scalar and matrix Chernoff-style concentration bounds for a broad class of probability distributions over the binary hypercube $\{0,1\}^n$. Motivated by recent tools developed for the study of mixing times of Markov chains on…
How does the information flow between different brain regions during various stimuli? This is the question we aim to address by studying complex cognitive paradigms in terms of Information Theory. To assess creativity and the emergence of…
Ribbons are a class of slender structures whose length, width, and thickness are widely separated from each other. This scale separation gives a ribbon unusual mechanical properties in athermal macroscopic settings, e.g. it can bend without…
We introduce a framework for obtaining tight mixing times for Markov chains based on what we call restricted modified log-Sobolev inequalities. Modified log-Sobolev inequalities (MLSI) quantify the rate of relative entropy contraction for…
Fisher information, Shannon information entropy and Statistical Complexity are calculated for the interface of a normal metal and a superconductor, as a function of the temperature for several materials. The order parameter $\Psi({\bf r})$…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
The statistical mechanics of a ribbon polymer made up of two semiflexible chains is studied using both analytical techniques and simulation. The system is found to have a crossover transition at some finite temperature, from a type of short…
We derive independence tests by means of dependence measures thresholding in a semiparametric context. Precisely, estimates of phi-mutual informations, associated to phi-divergences between a joint distribution and the product distribution…
Recent works investigated the generalization properties in deep neural networks (DNNs) by studying the Information Bottleneck in DNNs. However, the mea- surement of the mutual information (MI) is often inaccurate due to the density…
We continue the study of the quantum marginal independence problem, namely the question of which faces of the subadditivity cone are achievable by quantum states. We introduce a new representation of the patterns of marginal independence…
We study the independence structure of finitely exchangeable distributions over random vectors and random networks. In particular, we provide necessary and sufficient conditions for an exchangeable vector so that its elements are completely…
We study how to establish $\textit{spectral independence}$, a key concept in sampling, without relying on total influence bounds, by applying an $\textit{approximate inverse}$ of the influence matrix. Our method gives constant upper bounds…
We establish an exact identity for overdamped Langevin dynamics: the total entropy production rate equals four times the mutual information rate between an infinitesimal displacement and its time midpoint, plus a mean flow term. This yields…