Related papers: Remarks on the Most Informative Function Conjectur…
We study two conjectures posed in the analysis of Boolean functions $f : \{-1, 1\}^n \to \{-1, 1\}$, in both of which, the Majority function plays a central role: the "Majority is Least Stable" (Benjamini et al., 1999) and the…
We characterize mutual information as the unique map on ordered pairs of random variables satisfying a set of axioms similar to those of Faddeev's characterization of the Shannon entropy. There is a new axiom in our characterization however…
The Standard Simplex Conjecture of Isaksson and Mossel asks for the partition $\{A_{i}\}_{i=1}^{k}$ of $\mathbb{R}^{n}$ into $k\leq n+1$ pieces of equal Gaussian measure of optimal noise stability. That is, for $\rho>0$, we maximize $$…
To understand the sample-to-sample fluctuations in disorder-generated multifractal patterns we investigate analytically as well as numerically the statistics of high values of the simplest model - the ideal periodic $1/f$ Gaussian noise. By…
We study the secrecy capacity in the vicinity of colluding eavesdroppers. Contrary to the perfect collusion assumption in previous works, our new information-theoretic model considers constraints in collusion. We derive the achievable…
Estimating mutual information between continuous random variables is often intractable and extremely challenging for high-dimensional data. Recent progress has leveraged neural networks to optimize variational lower bounds on mutual…
We study formal properties of smooth max-information, a generalization of von Neumann mutual information derived from the max-relative entropy. Recent work suggests that it is a useful quantity in one-shot channel coding, quantum rate…
A fundamental question in causal inference is whether it is possible to reliably infer manipulation effects from observational data. There are a variety of senses of asymptotic reliability in the statistical literature, among which the most…
In this work we present a new method for the estimation of Mutual Information (MI) between random variables. Our approach is based on an original interpretation of the Girsanov theorem, which allows us to use score-based diffusion models to…
Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual…
The conditional mutual information I(X;Y|Z) measures the average information that X and Y contain about each other given Z. This is an important primitive in many learning problems including conditional independence testing, graphical model…
A privacy-constrained information extraction problem is considered where for a pair of correlated discrete random variables $(X,Y)$ governed by a given joint distribution, an agent observes $Y$ and wants to convey to a potentially public…
A well-known discovery of Feige's is the following: Let $X_1, \ldots, X_n$ be nonnegative independent random variables, with $\mathbb{E}[X_i] \leq 1 \;\forall i$, and let $X = \sum_{i=1}^n X_i$. Then for any $n$, \[\Pr[X < \mathbb{E}[X] +…
We give a general proof of the strong consistency of the Maximum Likelihood Estimator for the case of independent non-identically distributed (i.n.i.d) data, assuming that the density functions of the random variables follow a particular…
We consider mean squared estimation with lookahead of a continuous-time signal corrupted by additive white Gaussian noise. We show that the mutual information rate function, i.e., the mutual information rate as function of the…
We derive the property of strong superadditivity of mutual information arising from the Markov property of the vacuum state in a conformal field theory and strong subadditivity of entanglement entropy. We show this inequality encodes…
Jaynes' information theory formalism of statistical mechanics is applied to the stationary states of open, non-equilibrium systems. The key result is the construction of the probability distribution for the underlying microscopic phase…
We consider a variant of the classical notion of noise on the Boolean hypercube which gives rise to a new approach to inequalities regarding noise stability. We use this approach to give a new proof of the Majority is Stablest theorem by…
We study the pointwise supremum of convex integral functionals $\mathcal{I}_{f,\gamma}(\xi)= \sup_{Q} \left( \int_\Omega f(\omega,\xi(\omega))Q(d\omega)-\gamma(Q)\right)$ on $L^\infty(\Omega,\mathcal{F},\mathbb{P})$ where…
We postulate the existence of a universal uncertainty relation between the quantum and classical mutual informations between pairs of quantum systems. Specifically, we propose that the sum of the classical mutual information, determined by…