Related papers: Some Information Inequalities for Statistical Infe…
We provide a general constrained risk inequality that applies to arbitrary non-decreasing losses, extending a result of Brown and Low [Ann. Stat. 1996]. Given two distributions $P_0$ and $P_1$, we find a lower bound for the risk of…
We present a general approach, based on exponential inequalities, to derive bounds on the generalization error of randomized learning algorithms. Using this approach, we provide bounds on the average generalization error as well as bounds…
Parametrized families of density operators are studied. A generalization of the lower bound of Cramer and Rao is formulated. It involves escort density operators. The notion of phi-exponential family is introduced. This family, together…
We consider two variants of a quantum-statistical generalization of the Cramer-Rao inequality that establishes an invariant lower bound on the mean square error of a generalized quantum measurement. The proposed complex variant of this…
We present new information-theoretic generalization guarantees through the a novel construction of the "neighboring-hypothesis" matrix and a new family of stability notions termed sample-conditioned hypothesis (SCH) stability. Our approach…
Statistical Inference is the process of determining a probability distribution over the space of parameters of a model given a data set. As more data becomes available this probability distribution becomes updated via the application of…
Often of primary interest in the analysis of multivariate data are the copula parameters describing the dependence among the variables, rather than the univariate marginal distributions. Since the ranks of a multivariate dataset are…
We survey recent work on the Cramer-Rao inequality by Hasagawa and Petz; the notion of information manifold in infinite-dimensional Hilbert spaces is introduced, and the extension by Grasselli and the author to quadratic form perturbations…
Using the generalized entropies which depend on two parameters we propose a set of quantitative characteristics derived from the Information Geometry based on these entropies. Our aim, at this stage, is modest, as we are first constructing…
The variance and the entropy power of a continuous random variable are bounded from below by the reciprocal of its Fisher information through the Cram\'{e}r-Rao bound and the Stam's inequality respectively. In this note, we introduce the…
The Cramer-Rao-Frechet inequality is reviewed specializing it to track fitting. A diffused opinion attributes to this inequality the limitation of the resolution of the track fits with the number N of observations. It turns out that this…
Under minimal regularity assumptions, we establish a family of information-theoretic Bayesian Cram\'er-Rao bounds, indexed by probability measures that satisfy a logarithmic Sobolev inequality. This family includes as a special case the…
The Bayesian Cram\'er-Rao bound (CRB) provides a lower bound on the mean square error of any Bayesian estimator under mild regularity conditions. It can be used to benchmark the performance of statistical estimators, and provides a…
A general class of Bayesian lower bounds when the underlying loss function is a Bregman divergence is demonstrated. This class can be considered as an extension of the Weinstein--Weiss family of bounds for the mean squared error and relies…
The Cramer-Rao product of the Fisher information and the variance of a probability density \rho(x), defined on a domain \Delta \in R^D, is found to have a minimum value reached by the density associated with the ground state of the harmonic…
We derive lower bounds on the Bayes risk in decentralized estimation, where the estimator does not have direct access to the random samples generated conditionally on the random parameter of interest, but only to the data received from…
We introduce an inequality which may be viewed as a generalization of both the Brascamp-Lieb inequality and its reverse (Barthe's inequality), and prove its information-theoretic (i.e.\ entropic) formulation. This result leads to a unified…
This is a short survey on existing upper and lower bounds on the probability of the union of a finite number of events using partial information given in terms of the individual or pairwise event probabilities (or their sums). New proofs…
The task of parametric model selection is cast in terms of a statistical mechanics on the space of probability distributions. Using the techniques of low-temperature expansions, we arrive at a systematic series for the Bayesian posterior…
The present paper offers, in its first part, a unified approach for the derivation of families of inequalities for set functions which satisfy sub/supermodularity properties. It applies this approach for the derivation of information…