Related papers: Pointwise Relations between Information and Estima…
Recent advances in statistical learning theory have revealed profound connections between mutual information (MI) bounds, PAC-Bayesian theory, and Bayesian nonparametrics. This work introduces a novel mutual information bound for…
Estimation of a deterministic quantity observed in non-Gaussian additive noise is explored via order statistics approach. More specifically, we study the estimation problem when measurement noises either have positive supports or follow a…
Traditional meta-analysis assumes that the effect sizes estimated in individual studies follow a Gaussian distribution. However, this distributional assumption is not always satisfied in practice, leading to potentially biased results. In…
The emergence of quantum technologies has brought much attention to the characterization of quantum resources as well as the classical simulatability of quantum processes. Quantum resources, as quantified by non-stabilizerness, have in one…
Conventionally, covariances do not distinguish between spatial and temporal correlations. The same covariance matrix could equally describe temporal correlations between observations of the same system at two different times or correlations…
We study the problem of common randomness (CR) generation in the basic two-party communication setting in which the sender and the receiver aim to agree on a common random variable with high probability by observing independent and…
A complete first and second order statistical characterization of noise in SENSE reconstructed data is proposed. SENSE acquisitions have usually been modeled as Rician distributed, since the data reconstruction takes place into the spatial…
We introduce an information-theoretic quantity with similar properties to mutual information that can be estimated from data without making explicit assumptions on the underlying distribution. This quantity is based on a recently proposed…
We propose a novel approach to the problem of mutual information (MI) estimation via introducing a family of estimators based on normalizing flows. The estimator maps original data to the target distribution, for which MI is easier to…
We propose a novel estimator of the mutual information between two ordinal vectors $x$ and $y$. Our approach is inductive (as opposed to deductive) in that it depends on the data generating distribution solely through some nonparametric…
We present a new notion $R_\ell$ of higher-order common information, which quantifies the information that $\ell\geq 2$ arbitrarily distributed random variables have in common. We provide analytical lower bounds on $R_3$ and $R_4$ for…
An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of…
A representation of Gaussian distributed sparsely sampled longitudinal data in terms of predictive distributions for their functional principal component scores (FPCs) maps available data for each subject to a multivariate Gaussian…
I calculate the statistics of correlation of two digitized noiselike signals, which are drawn from complex Gaussian distributions, sampled, quantized, correlated, and averaged. Averaged over many such samples, the correlation r approaches a…
A formula is derived for the capacity of the Gaussian channel with a benevolent message-cognizant rate-limited helper that provides a noncausal description of the noise to the encoder and decoder. This capacity is strictly larger than when…
Mutual information (MI) is a fundamental measure of statistical dependence between two variables, yet accurate estimation from finite data remains notoriously difficult. No estimator is universally reliable, and common approaches fail in…
The feedback capacity of the stationary Gaussian additive noise channel has been open, except for the case where the noise is white. Here we find the feedback capacity of the stationary first-order moving average additive Gaussian noise…
The problem of fast point-to-point MIMO channel mutual information estimation is addressed, in the situation where the receiver undergoes unknown colored interference, whereas the channel with the transmitter is perfectly known. The…
We study data processing inequalities that are derived from a certain class of generalized information measures, where a series of convex functions and multiplicative likelihood ratios are nested alternately. While these information…
A simple model for image formation in linear shift-invariant systems is considered, in which both the detected signal and the noise variance are varying slowly compared to the point-spread function of the system. It is shown that within the…