Related papers: On mutual information, likelihood-ratios and estim…
The Amosov-Holevo-Werner conjecture implies the additivity of the minimum Re'nyi entropies at the output of a channel. The conjecture is proven true for all Re'nyi entropies of integer order greater than two in a class of Gaussian bosonic…
The Fisher Information Matrix formalism is extended to cases where the data is divided into two parts (X,Y), where the expectation value of Y depends on X according to some theoretical model, and X and Y both have errors with arbitrary…
Estimation of mutual information between (multidimensional) real-valued variables is used in analysis of complex systems, biological systems, and recently also quantum systems. This estimation is a hard problem, and universally good…
Mutual information is a general statistical dependency measure which has found applications in representation learning, causality, domain generalization and computational biology. However, mutual information estimators are typically…
In this paper, we first establish general bounds on the Fisher information distance to the class of normal distributions of Malliavin differentiable random variables. We then study the rate of Fisher information convergence in the central…
The effect of "good", point-to-point capacity achieving, code sequences on an additional signal, of bounded variance, transmitted over the additive Gaussian noise channel is examined. For such code sequences, it is shown that their effect,…
We present two classes of improved estimators for mutual information $M(X,Y)$, from samples of random points distributed according to some joint probability density $\mu(x,y)$. In contrast to conventional estimators based on binnings, they…
Recent advances in statistical learning theory have revealed profound connections between mutual information (MI) bounds, PAC-Bayesian theory, and Bayesian nonparametrics. This work introduces a novel mutual information bound for…
We study the mutual information estimation for mixed-pair random variables. One random variable is discrete and the other one is continuous. We develop a kernel method to estimate the mutual information between the two random variables. The…
Factorizing low-rank matrices has many applications in machine learning and statistics. For probabilistic models in the Bayes optimal setting, a general expression for the mutual information has been proposed using heuristic statistical…
An empirical Bayes approach to the estimation of possibly sparse sequences observed in Gaussian white noise is set out and investigated. The prior considered is a mixture of an atom of probability at zero and a heavy-tailed density \gamma,…
Given finite-dimensional random vectors $Y$, $X$, and $Z$ that form a Markov chain in that order (i.e., $Y \to X \to Z$), we derive upper bounds on the excess minimum risk using generalized information divergence measures. Here, $Y$ is a…
In this work, we consider the problem of distributed approximation of functions over multiple-access channels with additive noise. In contrast to previous works, we take fast fading into account and give explicit probability bounds for the…
We consider the problem of estimating means of two Gaussians in a 2-Gaussian mixture, which is not balanced and is corrupted by noise of an arbitrary distribution. We present a robust algorithm to estimate the parameters, together with…
The inverse relation between mutual information (MI) and Bayesian error is sharpened by deriving finite sequences of upper and lower bounds on MI in terms of the minimum probability of error (MPE) and related Bayesian quantities. The well…
The conditional mean is a fundamental and important quantity whose applications include the theories of estimation and rate-distortion. It is also notoriously difficult to work with. This paper establishes novel bounds on the differential…
In this letter, we consider two sets of observations defined as subspace signals embedded in noise and we wish to analyze the distance between these two subspaces. The latter entails evaluating the angles between the subspaces, an issue…
Gravitational-wave astronomers often wish to characterize the expected parameter-estimation accuracy of future observations. The Fisher matrix provides a lower bound on the spread of the maximum-likelihood estimator across noise…
In this paper we analyse the behaviour of adaptive filters or detectors when they are trained with $t$-distributed samples rather than Gaussian distributed samples. More precisely we investigate the impact on the distribution of some…
We study secure source-coding with causal disclosure, under the Gaussian distribution. The optimality of Gaussian auxiliary random variables is shown in various scenarios. We explicitly characterize the tradeoff between the rates of…