Related papers: Mutual Information Approximation
Mutual Information (MI) and Conditional Mutual Information (CMI) are multi-purpose tools from information theory that are able to naturally measure the statistical dependencies between random variables, thus they are usually of central…
Rectified Flow (RF) models trained with a Flow matching framework have achieved state-of-the-art performance on Text-to-Image (T2I) conditional generation. Yet, multiple benchmarks show that synthetic images can still suffer from poor…
Mutual information (MI) minimization has gained considerable interests in various machine learning tasks. However, estimating and minimizing MI in high-dimensional spaces remains a challenging problem, especially when only samples, rather…
We study generalized free fields (GFF) from the point of view of information measures. We first review conformal GFF, their holographic representation, and the ambiguities in the assignation of algebras to regions that arise in these…
We consider a mutual information (MI) regularized version of optimal density control of a discrete-time linear system. MI optimal control has been proposed as an extension of maximum entropy optimal control to trade off between control…
Consider a pair of terminals connected by two independent additive white Gaussian noise channels, and limited by individual power constraints. The first terminal would like to reliably send information to the second terminal, within a given…
Constellation shaping is a well-established method to improve upon a regular quadrature amplitude modulation (QAM). It is known that the gain achieved by any shaping method for an additive white Gaussian noise (AWGN) channel is…
Millimeter-wave (mmWave) massive Multiple Input Multiple Output (MIMO) systems encounter both spatial wideband spreading and temporal wideband effects in the communication channels of individual users. Accurate estimation of a user's…
Broadband frequency-selective fading channels usually have the inherent sparse nature. By exploiting the sparsity, adaptive sparse channel estimation (ASCE) methods, e.g., reweighted L1-norm least mean square (RL1-LMS), could bring a…
We consider a multiple-input multiple-output (MIMO) multiple access channel (MAC), where the channel between each transmitter and the receiver is modeled by the doubly-scattering channel model. Based on novel techniques from random matrix…
Variational approaches based on neural networks are showing promise for estimating mutual information (MI) between high dimensional variables. However, they can be difficult to use in practice due to poorly understood bias/variance…
Numerical upper and lower bounds to the information rate transferred through the additive white Gaussian noise channel affected by discrete-time multiplicative autoregressive moving-average (ARMA) phase noise are proposed in the paper. The…
Understanding dependencies between variables is critical for interpretability and efficient generation in masked diffusion models (MDMs), yet these models primarily expose marginal conditional distributions and do not explicitly represent…
The main difficulty concerning optimizing the mutual information (MI) in reconfigurable intelligent surface (RIS)-aided communication systems with discrete signaling is the inability to formulate this optimization problem in an analytically…
This work focuses on optimizing the hybrid quantum noise model to improve the capacity of Gaussian quantum channels using Machine Learning (ML) generated clusters. The work specifically leverages Gaussian Mixture Model (GMM) and the…
We propose the exact calculation of the probability density function (PDF) and cumulative distribution function (CDF) of mutual information (MI) for a two-user MIMO MAC network over block Rayleigh fading channels. So far the PDF and CDF…
End-to-end deep learning for communication systems, i.e., systems whose encoder and decoder are learned, has attracted significant interest recently, due to its performance which comes close to well-developed classical encoder-decoder…
This paper presents novel information-theoretic PHY layer performance improvement for combating the system fading effects in wireless terrestrial communication applications. Robust space-time system strategies in cooperation with the high…
The entropy computation of Gaussian mixture distributions with a large number of components has a prohibitive computational complexity. In this paper, we propose a novel approach exploiting the sphere decoding concept to bound and…
The statistical analysis of the fading End to End channel gain of the amplify and forward relaying protocol, however, is well known as extremely complex, and research work to date have only studied the asymptotic behaviour of the outage…