Related papers: Efron-Stein PAC-Bayesian Inequalities

Virtually any model we use in machine learning to make predictions does not perfectly represent reality. So, most of the learning happens under model misspecification. In this work, we present a novel analysis of the generalization…

We establish nonuniform Berry-Esseen bounds for martingales under the conditional Bernstein condition. These bounds imply Cram\'er type large deviations for moderate $x$'s, and are of exponential decay rate as de la Pe\~na's inequality when…

Operator-valued concentration inequalities are foundational to the analysis of modern high-dimensional statistics and randomized algorithms. However, standard oracle bounds are frequently limited in practice: they require explicit a priori…

Approximate Bayesian inference methods provide a powerful suite of tools for finding approximations to intractable posterior distributions. However, machine learning applications typically involve selecting actions, which -- in a Bayesian…

Let $T$ be a general sampling statistic that can be written as a linear statistic plus an error term. Uniform and non-uniform Berry--Esseen type bounds for $T$ are obtained. The bounds are the best possible for many known statistics.…

The core of generalization theory was developed for independent observations. Some PAC and PAC-Bayes bounds are available for data that exhibit a temporal dependence. However, there are constants in these bounds that depend on properties of…

We derive explicit central moment inequalities for random variables that admit a Stein coupling, such as exchangeable pairs, size--bias couplings or local dependence, among others. The bounds are in terms of moments (not necessarily…

The topics dicussed in this paper take their origin inthe estimation of the Gram matrix of a random vector from a sample made of n independent copies. They comprise the estimation of the covariance matrix and the study of least squares…

We make three related contributions motivated by the challenge of training stochastic neural networks, particularly in a PAC-Bayesian setting: (1) we show how averaging over an ensemble of stochastic neural networks enables a new class of…

We develop a semiparametric Bayesian approach for estimating the mean response in a missing data model with binary outcomes and a nonparametrically modelled propensity score. Equivalently we estimate the causal effect of a treatment,…

PAC-Bayesian bounds are known to be tight and informative when studying the generalization ability of randomized classifiers. However, they require a loose and costly derandomization step when applied to some families of deterministic…

We prove concentration inequalities of the form $P(Y \ge t) \le \exp(-B(t))$ for a random variable $Y$ with mean zero and variance $\sigma^2$ using a coupling technique from Stein's method that is so-called approximate zero bias couplings.…

We provide a comprehensive semi-parametric study of Bayesian partially identified econometric models. While the existing literature on Bayesian partial identification has mostly focused on the structural parameter, our primary focus is on…

We revisit empirical Bayes discrimination detection, focusing on uncertainty arising from both partial identification and sampling variability. While prior work has mostly focused on partial identification, we find that some empirical…

Generalization is a central concept in machine learning theory, yet for quantum models, it is predominantly analyzed through uniform bounds that depend on a model's overall capacity rather than the specific function learned. These…

We obtain non-uniform Edgeworth expansions for several classes of weakly dependent (non-stationary) sequences of random variables, including uniformly elliptic inhomogeneous Markov chains, random and time-varying (partially) hyperbolic or…

We present algorithms for estimating the forward reachable set of a dynamical system using only a finite collection of independent and identically distributed samples. The produced estimate is the sublevel set of a function called an…

We propose a Bayesian nonparametric model for mixed-type bounded data, where some variables are compositional and others are interval-bounded. Compositional variables are non-negative and sum to a given constant, such as the proportion of…

Pac-Bayes bounds are among the most accurate generalization bounds for classifiers learned from independently and identically distributed (IID) data, and it is particularly so for margin classifiers: there have been recent contributions…

Stein's method for concentration inequalities was introduced to prove concentration of measure in problems involving complex dependencies such as random permutations and Gibbs measures. In this paper, we provide some extensions of the…