Related papers: Some Information Inequalities for Statistical Infe…

Causal parameters may not be point identified in the presence of unobserved confounding. However, information about non-identified parameters, in the form of bounds, may still be recovered from the observed data in some cases. We develop a…

We consider distributed parameter estimation using interactive protocols subject to local information constraints such as bandwidth limitations, local differential privacy, and restricted measurements. We provide a unified framework…

A proof of the Cram\'er-Rao inequality for prediction is presented under conditions of $L^2$-differentiability of the family of distributions of the model. The assumptions and the proof differ from those of Miyata (2001) who also proved…

We present a unifying picture of PAC-Bayesian and mutual information-based upper bounds on the generalization error of randomized learning algorithms. As we show, Tong Zhang's information exponential inequality (IEI) gives a general recipe…

We correct claims about lower bounds on mutual information (MI) between real-valued random variables made in A. Kraskov {\it et al.}, Phys. Rev. E {\bf 69}, 066138 (2004). We show that non-trivial lower bounds on MI in terms of linear…

Effective bounds on the union probability are well known to be beneficial in the analysis of stochastic problems in many areas, including probability theory, information theory, statistical communications, computing and operations research.…

Information theory provides tools to predict the performance of a learning algorithm on a given dataset. For instance, the accuracy of learning an unknown parameter can be upper bounded by reducing the learning task to hypothesis testing…

Van Trees inequality, also known as the Bayesian Cram\'er-Rao lower bound, is a powerful tool for establishing lower bounds for minimax estimation through Fisher information. It easily adapts to different statistical models and often yields…

A generalised notion of exponential families is introduced. It is based on the variational principle, borrowed from statistical physics. It is shown that inequivalent generalised entropy functions lead to distinct generalised exponential…

The Cramer-Rao bound, satisfied by classical Fisher information, a key quantity in information theory, has been shown in different contexts to give rise to the Heisenberg uncertainty principle of quantum mechanics. In this paper, we show…

In this paper we derive sharp lower and upper bounds for the covariance of two bounded random variables when knowledge about their expected values, variances or both is available. When only the expected values are known, our result can be…

The Cram\'er-Rao bound (CRB), a well-known lower bound on the performance of any unbiased parameter estimator, has been used to study a wide variety of problems. However, to obtain the CRB, requires an analytical expression for the…

We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…

The thermodynamic uncertainty relation is an inequality stating that it is impossible to attain higher precision than the bound defined by entropy production. In statistical inference theory, information inequalities assert that it is…

We prove lower bounds on the error of any estimator for the mean of a real probability distribution under the knowledge that the distribution belongs to a given set. We apply these lower bounds both to parametric and nonparametric…

We present a new method to propagate lower bounds on conditional probability distributions in conventional Bayesian networks. Our method guarantees to provide outer approximations of the exact lower bounds. A key advantage is that we can…

We present deviation bounds for self-normalized averages and applications to estimation with a random number of observations. The results rely on a peeling argument in exponential martingale techniques that represents an alternative to the…

We evaluate generalized information measures constructed with Husimi distributions and connect them with the Wehrl entropy, on the one hand, and with thermal uncertainty relations, on the other one. The concept of escort distribution plays…

We derive an asymptotic lower bound on the Bayes risk when N identical quantum systems whose state depends on a vector of unknown parameters are jointly measured in an arbitrary way and the parameters of interest estimated on the basis of…

Large-scale datasets are increasingly being used to inform decision making. While this effort aims to ground policy in real-world evidence, challenges have arisen as selection bias and other forms of distribution shifts often plague…