Related papers: Extensions of the I-MMSE Relation
Relative Fisher information, also known as score matching, is a recently introduced learning method for parameter estimation. Fundamental relations between relative entropy and score matching have been established in the literature for…
In this work we study the capacity of interference-limited channels with memory. These channels model non-orthogonal communications scenarios, such as the non-orthogonal multiple access (NOMA) scenario and underlay cognitive communications,…
In this paper, we introduce new Stein identities for gamma target distribution as well as a new non-linear channel specifically designed for gamma inputs. From these two ingredients, we derive an explicit and simple formula for the…
We study a dephasing channel with memory, modelled by a multimode environment of oscillators. Focusing on the case of two channel uses, we show that memory effects can enhance the amount of coherent quantum information transmitted down the…
The operational capacity of Gaussian MIMO channels with memory was obtained by Brandenburg and Wyner in [9] under certain mild assumptions on the channel impulse response and its noise covariance matrix, which essentuially require channel…
We propose a channel modeling using jump-diffusion processes, and study the differential properties of entropy and mutual information. By utilizing the Kramers-Moyal and Kolmogorov-Feller equations, we express the mutual information between…
We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections, a problem relevant in compressed sensing, sparse superposition codes or code division multiple access just to cite few. There has…
Mutual information (MI) is one of the most general ways to measure relationships between random variables, but estimating this quantity for complex systems is challenging. Denoising diffusion models have recently set a new bar for density…
We continue to discuss why MMSE estimation arises in coding schemes that approach the capacity of linear Gaussian channels. Here we consider schemes that involve successive decoding, such as decision-feedback equalization or successive…
Mediation analysis is appealing for its ability to improve understanding of the mechanistic drivers of causal effects, but real-world data complexities challenge its successful implementation, including: 1) the existence of post-exposure…
We investigate how to exploit intermittent feedback for interference management by studying the two-user Gaussian interference channel (IC). We approximately characterize (within a universal constant) the capacity region for the Gaussian IC…
An abstraction for multisensor communication termed the Gaussian Multiplex Channel is presented and analyzed. In this model, the sensor outputs can be added together in any combination through a network of switches, and the combinations can…
We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning models. To accurately estimate MI from a finite number of samples, we present GMM-MI…
We consider the problem of signal estimation (denoising) from a statistical mechanical perspective, using a relationship between the minimum mean square error (MMSE), of estimating a signal, and the mutual information between this signal…
In this paper, we generalize the fundamental relation between the derivative of the mutual information and the minimum mean squared error (MMSE) to multiuser setups. We prove that the derivative of the mutual information with respect to the…
This paper investigates a channel estimator based on Gaussian mixture models (GMMs) in the context of linear inverse problems with additive Gaussian noise. We fit a GMM to given channel samples to obtain an analytic probability density…
We extend the recent analysis of the rate-reliability tradeoff in deterministic identification (DI) to general linear Gaussian channels, marking the first such analysis for channels with continuous output. Because DI provides a framework…
We consider mean squared estimation with lookahead of a continuous-time signal corrupted by additive white Gaussian noise. We show that the mutual information rate function, i.e., the mutual information rate as function of the…
We prove an exact relationship between the optimal denoising function and the data distribution in the case of additive Gaussian noise, showing that denoising implicitly models the structure of data allowing it to be exploited in the…
Taking a functional approach, we derive a general expression for the gradient of the Mutual Information (MI) with respect to the system parameters in the stochastic systems. This expression covers the cases in which the system input depends…