Related papers: Ergodic MIMO Mutual Information: Twenty Years Afte…
Estimating mutual information (MI) is a fundamental yet challenging task in data science and machine learning. This work proposes a new estimator for mutual information. Our main discovery is that a preliminary estimate of the data…
We introduce an information-theoretic quantity with similar properties to mutual information that can be estimated from data without making explicit assumptions on the underlying distribution. This quantity is based on a recently proposed…
Mutual information is a widely-used information theoretic measure to quantify the amount of association between variables. It is used extensively in many applications such as image registration, diagnosis of failures in electrical machines,…
We introduce the Mutual Information Machine (MIM), a novel formulation of representation learning, using a joint distribution over the observations and latent state in an encoder/decoder framework. Our key principles are symmetry and mutual…
We argue that the estimation of mutual information between high dimensional continuous random variables can be achieved by gradient descent over neural networks. We present a Mutual Information Neural Estimator (MINE) that is linearly…
We prove the Courtade-Kumar conjecture, which states that the mutual information between any Boolean function of an $n$-dimensional vector of independent and identically distributed inputs to a memoryless binary symmetric channel and the…
Estimating mutual information between continuous random variables is often intractable and extremely challenging for high-dimensional data. Recent progress has leveraged neural networks to optimize variational lower bounds on mutual…
This paper studies the ergodic mutual information of hybrid selection/maximal-ratio combining (H-S/MRC) diversity system under BPSK/QPSK modulations. We consider a simple single-input multiple-output (SIMO) channel, where a subset of…
We explore a well-known integral representation of the logarithmic function, and demonstrate its usefulness in obtaining compact, easily-computable exact formulas for quantities that involve expectations and higher moments of the logarithm…
We prove the Courtade-Kumar conjecture, for several classes of n-dimensional Boolean functions, for all $n \geq 2$ and for all values of the error probability of the binary symmetric channel, $0 \leq p \leq 1/2$. This conjecture states that…
This paper builds upon our recent work which computed the moment generating function of the MIMO mutual information exactly in terms of a Painlev\'{e} V differential equation. By exploiting this key analytical tool, we provide an in-depth…
Stacked intelligent metasurface (SIM) is a promising enabler for next-generation high-capacity networks that exhibit better performance compared to its single-layer counterpart by means of just wave propagation. However, the study of…
Mutual information (MI) is a promising candidate measure for the assessment and optimization of localization systems, as it captures nonlinear dependencies between random variables. However, the high cost of computing MI, especially for…
The Mutual Information (MI) is an often used measure of dependency between two random variables utilized in information theory, statistics and machine learning. Recently several MI estimators have been proposed that can achieve parametric…
Mutual information is used as a purely geometrical regularization of entanglement entropy applicable to any QFT. A coefficient in the mutual information between concentric circular entangling surfaces gives a precise universal prescription…
Modeling communication channels as thermal systems results in Hamiltonians which are an explicit function of the temperature. The first two authors have recently generalized the second thermodynamic law to encompass systems with…
The problem of fast point-to-point MIMO channel mutual information estimation is addressed, in the situation where the receiver undergoes unknown colored interference, whereas the channel with the transmitter is perfectly known. The…
This paper explores the emerging research direction of electromagnetic information theory (EIT), which aims to integrate traditional Shannon-based methodologies with physical consistency, particularly the electromagnetic properties of…
Consider a random non-centered multiple antenna radio transmission channel. Assume that the deterministic part of the channel is itself frequency selective, and that the random multipath part is represented by an ergodic stationary vector…
We study the problem of estimating a parametric discrete memoryless channel \( p(y \mid x; \boldsymbol{\theta}) \) when the transmitter selects its input distribution \( \pi \) to maximize mutual information under the true parameter \(…