Related papers: A constrained risk inequality for general losses
In classification, the de facto method for aggregating individual losses is the average loss. When the actual metric of interest is 0-1 loss, it is common to minimize the average surrogate loss for some well-behaved (e.g. convex) surrogate.…
A new distribution on (0, 1), generalized Log-Lindley distribution, is proposed by extending the Log-Lindley distribution. This new distribution is shown to be a weighted Log-Lindley distribution. Important probabilistic and statistical…
We establish a lower bound on the entropy of weighted sums of (possibly dependent) random variables $(X_1, X_2, \dots, X_n)$ possessing a symmetric joint distribution. Our lower bound is in terms of the joint entropy of $(X_1, X_2, \dots,…
We study the tradeoff between the statistical error and communication cost of distributed statistical estimation problems in high dimensions. In the distributed sparse Gaussian mean estimation problem, each of the $m$ machines receives $n$…
The purpose of this paper is to discuss empirical risk minimization when the losses are not necessarily bounded and may have a distribution with heavy tails. In such situations, usual empirical averages may fail to provide reliable…
This paper describes a flexible framework for generalized low-rank tensor estimation problems that includes many important instances arising from applications in computational imaging, genomics, and network analysis. The proposed estimator…
We investigate how basic probability inequalities can be extended to an imprecise framework, where (precise) probabilities and expectations are replaced by imprecise probabilities and lower/upper previsions. We focus on inequalities giving…
We consider parameter estimation in distributed networks, where each sensor in the network observes an independent sample from an underlying distribution and has $k$ bits to communicate its sample to a centralized processor which computes…
This paper proves, in very general settings, that convex risk minimization is a procedure to select a unique conditional probability model determined by the classification problem. Unlike most previous work, we give results that are general…
We consider a convex optimization problem with many linear inequality constraints. To deal with a large number of constraints, we provide a penalty reformulation of the problem, where the penalty is a variant of the one-sided Huber loss…
A risk analyst assesses potential financial losses based on multiple sources of information. Often, the assessment does not only depend on the specification of the loss random variable but also various economic scenarios. Motivated by this…
The aggregation of individual risks in large credit and insurance portfolios is guided by diversification and the law of large numbers, which formalizes the convergence of sample averages to their means. At the same time, regulatory capital…
The empirical loss, commonly referred to as the average loss, is extensively utilized for training machine learning models. However, in order to address the diverse performance requirements of machine learning models, the use of the…
We aim at estimating a function $\lambda:[0,1]\to \mathbb {R}$, subject to the constraint that it is decreasing (or increasing). We provide a unified approach for studying the $\mathbb {L}_p$-loss of an estimator defined as the slope of a…
The problem of determining the intrinsic quality of a signal processing system with respect to the inference of an unknown deterministic parameter $\theta$ is considered. While the Fisher information measure $F(\theta)$ forms a classical…
The problem of variable-rate lossless data compression is considered, for codes with and without prefix constraints. Sharp bounds are derived for the best achievable compression rate of memoryless sources, when the excess-rate probability…
We prove a new generalization bound that shows for any class of linear predictors in Gaussian space, the Rademacher complexity of the class and the training error under any continuous loss $\ell$ can control the test error under all Moreau…
Leave-one-out (LOO) prediction provides a principled, data-dependent measure of generalization, yet guarantees in fully transductive settings remain poorly understood beyond specialized models. We introduce Median of Level-Set Aggregation…
We give a distribution-dependent concentration inequality for functions of independent variables. The result extends Bernstein's inequality from sums to more general functions, whose variation in any argument does not depend too much on the…
This paper develops a general theory on rates of convergence of penalized spline estimators for function estimation when the likelihood functional is concave in candidate functions, where the likelihood is interpreted in a broad sense that…