Related papers: Exact upper and lower bounds on the misclassificat…
The existing upper and lower bounds between entropy and error probability are mostly derived from the inequality of the entropy relations, which could introduce approximations into the analysis. We derive analytical bounds based on the…
The exact lower bound on the probability of the occurrence of exactly one of $n$ random events each of probability $p$ is obtained.
In this paper, two new classes of lower bounds on the probability of error for $m$-ary hypothesis testing are proposed. Computation of the minimum probability of error which is attained by the maximum a-posteriori probability (MAP)…
A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…
We derive an (almost) guaranteed upper bound on the error of deep neural networks under distribution shift using unlabeled test data. Prior methods either give bounds that are vacuous in practice or give estimates that are accurate on…
Pattern recognition based on a high-dimensional predictor is considered. A classifier is defined which is based on a Transformer encoder. The rate of convergence of the misclassification probability of the classifier towards the optimal…
We discuss the problem of risk estimation in the classification problem, with specific focus on finding distributions that maximize the confidence intervals of risk estimation. We derived simple analytic approximations for the maximum bias…
New upper bounds on the relative entropy are derived as a function of the total variation distance. One bound refines an inequality by Verd\'{u} for general probability measures. A second bound improves the tightness of an inequality by…
Classification is a ubiquitous and fundamental problem in artificial intelligence and machine learning, with extensive efforts dedicated to developing more powerful classifiers and larger datasets. However, the classification task is…
Upper bounds on the Kolmogorov distance (and, equivalently in this case, on the total variation distance) between the Student distribution with p degrees of freedom (SD_p) and the standard normal distribution are obtained. These bounds are…
Lindsay and Basak (2000) posed the question of how far from normality could a distribution be if it matches $k$ normal moments. They provided a bound on the maximal difference in c.d.f.'s, and implied that these bounds were attained. It…
Error bounds based on worst likely assignments use permutation tests to validate classifiers. Worst likely assignments can produce effective bounds even for data sets with 100 or fewer training examples. This paper introduces a statistic…
This paper is concerned with estimating the intersection point of two densities, given a sample of both of the densities. This problem arises in classification theory. The main results provide lower bounds for the probability of the…
In statistics and machine learning, when we train a fitted model on available data, we typically want to ensure that we are searching within a model class that contains at least one accurate model -- that is, we would like to ensure an…
We show a direct relationship between the variance and the differential entropy for subclasses of symmetric and asymmetric unimodal distributions by providing an upper bound on variance in terms of entropy power. Combining this bound with…
This paper proposes an easy-to-compute upper bound for the overlap index between two probability distributions without requiring any knowledge of the distribution models. The computation of our bound is time-efficient and memory-efficient…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…
We present novel bounds for estimating discrete probability distributions under the $\ell_\infty$ norm. These are nearly optimal in various precise senses, including a kind of instance-optimality. Our data-dependent convergence guarantees…
Estimating the unknown number of classes in a population has numerous important applications. In a Poisson mixture model, the problem is reduced to estimating the odds that a class is undetected in a sample. The discontinuity of the odds…
The existing upper and lower bounds between entropy and error are mostly derived through an inequality means without linking to joint distributions. In fact, from either theoretical or application viewpoint, there exists a need to achieve a…