Related papers: Uniform Deviation Bounds for Unbounded Loss Functi…
We present a new family of information-theoretic generalization bounds within the framework of conditional mutual information (CMI). Most of our results are established based on the leave-$m$-out (L$m$O) cross-validation error, with $m$…
The density ratio is an important metric for evaluating the relative likelihood of two probability distributions, with extensive applications in statistics and machine learning. However, existing estimation theories for density ratios often…
Given finite-dimensional random vectors $Y$, $X$, and $Z$ that form a Markov chain in that order (i.e., $Y \to X \to Z$), we derive upper bounds on the excess minimum risk using generalized information divergence measures. Here, $Y$ is a…
Consider a nonparametric regression model with one-sided errors and regression function in a general H\"older class. We estimate the regression function via minimization of the local integral of a polynomial approximation. We show uniform…
Designing learning algorithms that are resistant to perturbations of the underlying data distribution is a problem of wide practical and theoretical importance. We present a general approach to this problem focusing on unsupervised…
We investigate the uniform convergence of subdifferential mappings from empirical risk to population risk in nonsmooth, nonconvex stochastic optimization. This question is key to understanding how empirical stationary points approximate…
We provide an elementary proof of the lower bound for the variance of continuous unimodal distributions and obtain analogous bounds for the higher order central moments. A lower bound for the rth central moment of discrete distribution is…
The purpose of the present paper is to establish explicit bounds on moderate deviation probabilities for a rather general class of geometric functionals enjoying the stabilization property, under Poisson input and the assumption of a…
We consider the problem of modulation and estimation of a random parameter $U$ to be conveyed across a discrete memoryless channel. Upper and lower bounds are derived for the best achievable exponential decay rate of a general moment of the…
The Median of Means (MoM) is a mean estimator that has gained popularity in the context of heavy-tailed data. In this work, we analyze its performance in the task of simultaneously estimating the mean of each function in a class…
We establish sharp upper and lower bounds for distortion risk metrics under distributional uncertainty. The uncertainty sets are characterized by four key features of the underlying distribution: mean, variance, unimodality, and Wasserstein…
Estimation of the complete distribution of a random variable is a useful primitive for both manual and automated decision making. This problem has received extensive attention in the i.i.d. setting, but the arbitrary data dependent setting…
In this paper, by proposing two new kinds of distributional uncertainty sets, we explore robustness of distortion risk measures against distributional uncertainty. To be precise, we first consider a distributional uncertainty set which is…
We obtain a denoising loss bound of the recently proposed neural network based universal discrete denoiser, Neural DUDE, which can adaptively learn its parameters solely from the noise-corrupted data, by minimizing the \emph{empirical…
A loss function measures the discrepancy between the true values and their estimated fits, for a given instance of data. In classification problems, a loss function is said to be proper if a minimizer of the expected loss is the true…
In this work, we deal with approximations for distribution functions of non-negative random variables. More specifically, we construct continuous approximants using an acceleration technique over a well-know inversion formula for Laplace…
In this paper, a uniform (over some parameter space) moment bound for the inverse of Fisher's information matrix is established. This result is then applied to develop moment bounds for the normalized least squares estimate in (nonlinear)…
We give improved constants for data dependent and variance sensitive confidence bounds, called empirical Bernstein bounds, and extend these inequalities to hold uniformly over classes of functionswhose growth function is polynomial in the…
This work provides test error bounds for iterative fixed point methods on linear predictors -- specifically, stochastic and batch mirror descent (MD), and stochastic temporal difference learning (TD) -- with two core contributions: (a) a…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…