Related papers: Fano's inequality for random variables
In this technical note, we give two extensions of the classical Fano inequality in information theory. The first extends Fano's inequality to the setting of estimation, providing lower bounds on the probability that an estimator of a…
In statistical inference problems, we wish to obtain lower bounds on the minimax risk, that is to bound the performance of any possible estimator. A standard technique to obtain risk lower bounds involves the use of Fano's inequality. In an…
Fano's inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano's inequality is generalized to a broad class of information measures, which contains those of Shannon…
Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning. While numerous…
Lower bounds involving $f$-divergences between the underlying probability measures are proved for the minimax risk in estimation problems. Our proofs just use simple convexity facts. Special cases and straightforward corollaries of our…
Fano's inequality reveals the relation between the conditional entropy and the probability of error . It has been the key tool in proving the converse of coding theorems in the past sixty years. In this paper, an extended Fano's inequality…
In this note, we present an information diffusion inequality derived from an elementary argument, which gives rise to a very general Fano-type inequality. The latter unifies and generalizes the distance-based Fano inequality and the…
We show how one can demonstrate continuous-variable Einstein-Podolsky-Rosen (EPR) steering without needing to characterize entire measurement probability distributions. To do this, we develop a modified Fano inequality useful for discrete…
We find the perhaps surprising inequality that the weighted average of independent and identically distributed Pareto random variables with infinite mean is larger than one such random variable in the sense of first-order stochastic…
We show an extension of Sanov's theorem on large deviations, controlling the tail probabilities of i.i.d. random variables with matching concentration and anti-concentration bounds. This result has a general scope, applies to samples of any…
Statistical inference for extreme values of random events is difficult in practice due to low sample sizes and inaccurate models for the studied rare events. If prior knowledge for extreme values is available, Bayesian statistics can be…
We present a generalization of the maximal inequalities that upper bound the expectation of the maximum of $n$ jointly distributed random variables. We control the expectation of a randomly selected random variable from $n$ jointly…
Lower bounds for the average probability of error of estimating a hidden variable X given an observation of a correlated random variable Y, and Fano's inequality in particular, play a central role in information theory. In this paper, we…
Interactive statistical decision making (ISDM) features algorithm-dependent data generated through interaction. Existing information-theoretic lower bounds in ISDM largely target expected risk, while tail-sensitive objectives are less…
We present some extensions of Bernstein's concentration inequality for random matrices. This inequality has become a useful and powerful tool for many problems in statistics, signal processing and theoretical computer science. The main…
We give a concentration inequality based on the premise that random variables take values within a particular region. The concentration inequality guarantees that, for any sequence of correlated random variables, the difference between the…
Estimation of the complete distribution of a random variable is a useful primitive for both manual and automated decision making. This problem has received extensive attention in the i.i.d. setting, but the arbitrary data dependent setting…
We provide an inequality which is a useful tool in studying both large deviation results and limit theorems for sums of random fields with "negligible" small values. In particular, the inequality covers cases of stable limits for random…
Let F be a distribution function with negative mean and regularly varying right tail. Under a mild smoothness condition we derive higher order asymptotic expansions for the tail distribution of the maxima of the random walk generated by F.…
Information theory provides tools to predict the performance of a learning algorithm on a given dataset. For instance, the accuracy of learning an unknown parameter can be upper bounded by reducing the learning task to hypothesis testing…