Related papers: Information-Theoretic Perspectives on Brascamp-Lie…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
Information theory, though originally developed for communications engineering, provides mathematical tools with broad applications across science. These tools characterize the fundamental limits of data compression and transmission in the…
While information is ubiquitously generated, shared, and analyzed in a modern-day life, there is still some controversy around the ways to asses the amount and quality of information inside a noisy optical channel. A number of theoretical…
Information diagram and the I-measure are useful mnemonics where random variables are treated as sets, and entropy and mutual information are treated as a signed measure. Although the I-measure has been successful in machine proofs of…
Efficient signal representation is essential for the functioning of living and artificial systems operating under resource constraints. A widely recognized framework for deriving such representations is the information bottleneck method,…
We consider two non-convex formulations for computing the optimal constant in the Brascamp-Lieb inequality corresponding to a given datum, and show that they are geodesically log-concave on the manifold of positive definite matrices endowed…
The optimal transport problem studies how to transport one measure to another in the most cost-effective way and has wide range of applications from economics to machine learning. In this paper, we introduce and study an information…
Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…
Vindicating a sophisticated but non-rigorous physics approach called the cavity method, we establish a formula for the mutual information in statistical inference problems induced by random graphs and we show that the mutual information…
A central challenge in statistical inference is the presence of confounding variables that may distort observed associations between treatment and outcome. Conventional "causal" methods, grounded in assumptions such as ignorability, exclude…
We are interested in learning data-driven representations that can generalize well, even when trained on inherently biased data. In particular, we face the case where some attributes (bias) of the data, if learned by the model, can severely…
In ordinary statistical mechanics the Boltzmann-Shannon entropy is related to the Maxwell-Bolzmann distribution $p_i$ by means of a twofold link. The first link is differential and is offered by the Jaynes Maximum Entropy Principle. The…
We study lower bounds on multilinear operators with Gaussian kernels acting on Lebesgue spaces, with exponents below one. We put forward natural conditions when the optimal constant can be computed by inspecting centered Gaussian functions…
The main topic of this thesis is the proof of two fundamental entropic inequalities for quantum Gaussian channels. Quantum Gaussian channels model the propagation of electromagnetic waves through optical fibers and free space in the quantum…
Forecasting techniques for assessing the power of future experiments to discriminate between theories or discover new laws of nature are of great interest in many areas of science. In this paper, we introduce a Bayesian forecasting method…
An inequality of Brascamp and Lieb provides a bound on the covariance of two functions with respect to log-concave measures. The bound estimates the covariance by the product of the $L^2$ norms of the gradients of the functions, where the…
In this paper, we analyze the fundamental trade-off between information transfer and power gain by means of an information-theoretic framework in communications circuits. This analysis is of interest as many of today's applications require…
The information bottleneck (IB) method offers an attractive framework for understanding representation learning, however its applications are often limited by its computational intractability. Analytical characterization of the IB method is…
We provide a number of new quantitative versions of Helly's theorem. For example, we show that for every family $\{P_i:i\in I\}$ of closed half-spaces $$P_i=\{x\in {\mathbb R}^n:\langle x,w_i\rangle \leq 1\}$$ in ${\mathbb R}^n$ such that…
In this paper, we take a unified approach for network information theory and prove a coding theorem, which can recover most of the achievability results in network information theory that are based on random coding. The final single-letter…