Related papers: Inequalities for the Bayes Risk
This note provides some new inequalities and approximations for beta distributions, including tail inequalities, exponential inequalities of Hoeffding and Bernstein type, Gaussian inequalities and approximations.
New upper and lower bounds for the error probability over an erasure channel are provided, making use of Wei's generalized weights, hierarchy and spectra. In many situations the upper and lower bounds coincide and this allows improvement of…
We provide a general constrained risk inequality that applies to arbitrary non-decreasing losses, extending a result of Brown and Low [Ann. Stat. 1996]. Given two distributions $P_0$ and $P_1$, we find a lower bound for the risk of…
In their seminal 1990 paper, Wasserman and Kadane establish an upper bound for the Bayes' posterior probability of a measurable set $A$, when the prior lies in a class of probability measures $\mathcal{P}$ and the likelihood is precise.…
This work presents several expected generalization error bounds based on the Wasserstein distance. More specifically, it introduces full-dataset, single-letter, and random-subset bounds, and their analogues in the randomized subsample…
We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data examples to reduce…
We extend a general Bernstein-type maximal inequality of Kevei and Mason (2011) for sums of random variables.
Bayesian classification labels observations based on given prior information, namely class-a priori and class-conditional probabilities. Bayes' risk is the minimum expected classification cost that is achieved by the Bayes' test, the…
We investigate the asymptotic behavior of parametric Bayes estimators under a broad class of loss functions that extend beyond the classical translation-invariant setting. To this end, we develop a unified theoretical framework for loss…
This paper provides a general technique for lower bounding the Bayes risk of statistical estimation, applicable to arbitrary loss functions and arbitrary prior distributions. A lower bound on the Bayes risk not only serves as a lower bound…
Generalization error bounds are essential to understanding machine learning algorithms. This paper presents novel expected generalization error upper bounds based on the average joint distribution between the output hypothesis and each…
We present a new PAC-Bayesian generalization bound. Standard bounds contain a $\sqrt{L_n \cdot \KL/n}$ complexity term which dominates unless $L_n$, the empirical error of the learning algorithm's randomized predictions, vanishes. We manage…
Certain new inequalities for the sums of factorials are presented.
We consider the concept of Bayes risk in the context of finite-dimensional ill-posed linear inverse problem with Gaussian prior and noise models. In this note, we rederive the following well-known result: in the present Gaussian linear…
Squared error loss remains the most commonly used loss function for constructing a Bayes estimator of the parameter of interest. However, it can lead to sub-optimal solutions when a parameter is defined in a restricted space. It can also be…
A generalized Cauchy-Schwarz inequality is derived and applied to uncertainty relation in quantum mechanics. We see a modification in the uncertainty relation and minimum uncertainty wave packet.
We derive new inequalities for the probabilities of projective measurements in mutually unbiased bases of a qudit system. These inequalities lead to wider ranges of validity and tighter bounds on entropic uncertainty inequalities previously…
Generalization bounds which assess the difference between the true risk and the empirical risk have been studied extensively. However, to obtain bounds, current techniques use strict assumptions such as a uniformly bounded or a Lipschitz…
Optimization is widely used in statistics, and often efficiently delivers point estimates on useful spaces involving structural constraints or combinatorial structure. To quantify uncertainty, Gibbs posterior exponentiates the negative loss…
The Bayes linear estimator is derived by minimizing the Bayes risk with respect to the squared loss function. Non-unbiased estimators such as ordinary ridge, typical shrinkage, fractional rank, and restricted least squares estimators, as…