Related papers: Inequalities for the Bayes Risk
We derive information-theoretic lower bounds on the Bayes risk and generalization error of realizable machine learning models. In particular, we employ an analysis in which the rate-distortion function of the model parameters bounds the…
Estimation of reliability and hazard rate is one of the most important problems raised in many applications especially in engineering studies as well as human lifetime. In this regard, different methods of estimation have been used. Each…
Some of the tightest information-theoretic generalization bounds depend on the average information between the learned hypothesis and a single training example. However, these sample-wise bounds were derived only for expected generalization…
We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of $f$-Divergences. The results are then applied to specific settings of interest and compared to other…
In random parameter estimation, Bayesian lower bounds (BLBs) for the mean-square error have been noticed to not be tight in a number of cases, even when the sample size, or the signal-to-noise ratio, grow to infinity. In this paper, we…
In this paper, we obtain the upper and lower bounds for two inequalities related to the range statistics. The first one is concerning the one-variable case and the second one is about the bivariate case.
We present a family of novel block-sample MAC-Bayes bounds (mean approximately correct). While PAC-Bayes bounds (probably approximately correct) typically give bounds for the generalization error that hold with high probability, MAC-Bayes…
Current PAC-Bayes generalisation bounds are restricted to scalar metrics of performance, such as the loss or error rate. However, one ideally wants more information-rich certificates that control the entire distribution of possible…
We prove general exponential moment inequalities for averages of [0,1]-valued iid random variables and use them to tighten the PAC Bayesian Theorem. The logarithmic dependence on the sample count in the enumerator of the PAC Bayesian bound…
A new least squares mean of positive definite matrices for the divergence associated with the sandwiched quasi-relative entropy has been introduced. It generalizes the well-known Wasserstein mean for covariance matrices of Gaussian…
Distributed consensus in the Wasserstein metric space of probability measures on the real line is introduced in this work. Convergence of each agent's measure to a common measure is proven under a weak network connectivity condition. The…
In this paper we investigate the estimation of the unknown parameters of a competing risk model based on a Weibull distributed decreasing failure rate and an exponentially distributed constant failure rate, under right censored…
In this work, we undertake a comprehensive reformulation, modification, and extension of Smith and Spiegelhalter's (1980) and (1982) Bayes Factor work within the evolving subject of Objective Bayes Factors. Our primary focus centers on…
We perform a quantitative analysis of the gain/loss asymmetry for financial time series by using a Bayesian approach. In particular, we focus on some selected indices and analyze the statistical significance of the asymmetry amount through…
We present new excess risk bounds for general unbounded loss functions including log loss and squared loss, where the distribution of the losses may be heavy-tailed. The bounds hold for general estimators, but they are optimized when…
We consider the problem of predicting as well as the best linear combination of d given functions in least squares regression, and variants of this problem including constraints on the parameters of the linear combination. When the input…
Many psychological theories can be operationalized as linear inequality constraints on the parameters of multinomial distributions (e.g., discrete choice analysis). These constraints can be described in two equivalent ways: Either as the…
New reverses of the Schwarz, triangle and Bessel inequalities in inner product spaces are pointed out. These results complement the recent ones obtained by the author in an earlier paper. Further, they are employed to establish new Gruss…
In this paper, we obtain a new generalization of Chebyshev's inequality for random elements taking values in a separate Banach space.
We prove semi-empirical concentration inequalities for random variables which are given as possibly nonlinear functions of independent random variables. These inequalities describe concentration of random variable in terms of the…