Related papers: Some Information Inequalities for Statistical Infe…
Traditional statistical learning theory relies on the assumption that data are identically and independently distributed (i.i.d.). However, this assumption often does not hold in many real-life applications. In this survey, we explore…
The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic. Tight bounds are obtained for U-statistics, smoothened L-statistics and…
The existing upper and lower bounds between entropy and error probability are mostly derived from the inequality of the entropy relations, which could introduce approximations into the analysis. We derive analytical bounds based on the…
Recent advances in statistical learning theory have revealed profound connections between mutual information (MI) bounds, PAC-Bayesian theory, and Bayesian nonparametrics. This work introduces a novel mutual information bound for…
We consider the problem of aggregating pairwise comparisons to obtain a consensus ranking order over a collection of objects. We use the popular Bradley-Terry-Luce (BTL) model which allows us to probabilistically describe pairwise…
In this work, we present a variety of novel information-theoretic generalization bounds for learning algorithms, from the supersample setting of Steinke & Zakynthinou (2020)-the setting of the "conditional mutual information" framework. Our…
Statistical models are used to produce estimates of demographic and global health indicators in populations with limited data. Such models integrate multiple data sources to produce estimates and forecasts with uncertainty based on model…
This paper provides a general technique for lower bounding the Bayes risk of statistical estimation, applicable to arbitrary loss functions and arbitrary prior distributions. A lower bound on the Bayes risk not only serves as a lower bound…
We revisit the problem of computing submatrices of the Cram\'er-Rao bound (CRB), which lower bounds the variance of any unbiased estimator of a vector parameter $\vth$. We explore iterative methods that avoid direct inversion of the Fisher…
We present a new proof of the quantum Cramer-Rao bound for precision parameter estimation [1-3] and extend it to a more general class of measurement procedures. We analyze a generalized framework for parameter estimation that covers most…
We propose the generalised Fisher information or the one-parameter extended class of the Fisher information for the case of one random variable. This new form of the Fisher information is obtained from the intriguing connection between the…
The necessary information for specifying a complex system may not be completely accessible to us, i.e., to mathematical treatments. This is not to be confounded with the incompleteness of our knowledge about whatever systems or nature,…
Classical probabilistic models of (noisy) quantum systems are not only relevant for understanding the non-classical features of quantum mechanics, but they are also useful for determining the possible advantage of using quantum resources…
We compute a variance lower bound for unbiased estimators in specified statistical models. The construction of the bound is related to the original Cram\'er-Rao bound, although it does not require the differentiability of the model.…
In this paper we develop the theory of information geometry for single random matrix models, with two goals: proving a Cramer-Rao theorem for estimators on random matrices, and calculating the Legendre transform of pressure and entropy with…
We present a new family of information-theoretic generalization bounds, in which the training loss and the population loss are compared through a jointly convex function. This function is upper-bounded in terms of the disintegrated,…
We introduce the notion of the essential tangent bundle of a parametrized measure model and the notion of reduced Fisher metric on a (possibly singular) 2-integrable measure model. Using these notions and a new characterization of…
We extend Fano's inequality, which controls the average probability of events in terms of the average of some $f$--divergences, to work with arbitrary events (not necessarily forming a partition) and even with arbitrary $[0,1]$--valued…
Determining the strength of non-linear statistical dependencies between two variables is a crucial matter in many research fields. The established measure for quantifying such relations is the mutual information. However, estimating mutual…
We derive information-theoretic converses (i.e., lower bounds) for the minimum time required by any algorithm for distributed function computation over a network of point-to-point channels with finite capacity, where each node of the…