Related papers: PAC-Bayes-Chernoff bounds for unbounded losses
Almost 10 years ago, Impagliazzo and Kabanets (2010) gave a new combinatorial proof of Chernoff's bound for sums of bounded independent random variables. Unlike previous methods, their proof is constructive. This means that it provides an…
We investigate the in-distribution generalization of machine learning algorithms. We depart from traditional complexity-based approaches by analyzing information-theoretic bounds that quantify the dependence between a learning algorithm and…
Identifying optimal values for a high-dimensional set of hyperparameters is a problem that has received growing attention given its importance to large-scale machine learning applications such as neural architecture search. Recently…
Under minimal regularity assumptions, we establish a family of information-theoretic Bayesian Cram\'er-Rao bounds, indexed by probability measures that satisfy a logarithmic Sobolev inequality. This family includes as a special case the…
The ultimate performance of machine learning algorithms for classification tasks is usually measured in terms of the empirical error probability (or accuracy) based on a testing dataset. Whereas, these algorithms are optimized through the…
Quantum Cram\'er--Rao theory is intrinsically local: it bounds precision near a specified parameter value, and its saturating measurement generally depends on that value. Barankin-type bounds use finite parameter displacements, but remain…
Many results in the quantum metrology literature use the Cram\'er-Rao bound and the Fisher information to compare different quantum estimation strategies. However, there are several assumptions that go into the construction of these tools,…
We study the problem of aggregation under the squared loss in the model of regression with deterministic design. We obtain sharp PAC-Bayesian risk bounds for aggregates defined via exponential weights, under general assumptions on the…
We develop a technique for deriving data-dependent error bounds for transductive learning algorithms based on transductive Rademacher complexity. Our technique is based on a novel general error bound for transduction in terms of…
The idea of the restricted mean has been used to establish a significantly improved version of Markov's inequality that does not require any new assumptions. The result immediately extends on Chebyshev's inequalities and Chernoff's bound.…
Bounding causal effects analytically, rather than numerically, is appealing for its interpretability and conceptual clarity. Existing sharp methods rely on optimization-based approaches such as the Balke-Pearl framework, whose computational…
We present a new method to propagate lower bounds on conditional probability distributions in conventional Bayesian networks. Our method guarantees to provide outer approximations of the exact lower bounds. A key advantage is that we can…
We study the issue of PAC-Bayesian domain adaptation: We want to learn, from a source domain, a majority vote model dedicated to a target one. Our theoretical contribution brings a new perspective by deriving an upper-bound on the target…
We present a framework to derive bounds on the test loss of randomized learning algorithms for the case of bounded loss functions. Drawing from Steinke & Zakynthinou (2020), this framework leads to bounds that depend on the conditional…
Standard Bayesian learning is known to have suboptimal generalization capabilities under misspecification and in the presence of outliers. PAC-Bayes theory demonstrates that the free energy criterion minimized by Bayesian learning is a…
A Chernoff-type distribution is a nonnormal distribution defined by the slope at zero of the greatest convex minorant of a two-sided Brownian motion with a polynomial drift. While a Chernoff-type distribution is known to appear as the…
Variational approximation techniques and inference for stochastic models in machine learning has gained much attention the last years. Especially in the case of Gaussian Processes (GP) and their deep versions, Deep Gaussian Processes…
We are motivated by the problem of performing failure prediction for safety-critical robotic systems with high-dimensional sensor observations (e.g., vision). Given access to a black-box control policy (e.g., in the form of a neural…
For independent $X$ and $Y$ in the inequality $P(X\leq Y+\mu)$, we give sharp lower bounds for unimodal distributions having finite variance, and sharp upper bounds assuming symmetric densities bounded by a finite constant. The lower bounds…
This paper presents an approach for learning vision-based planners that provably generalize to novel environments (i.e., environments unseen during training). We leverage the Probably Approximately Correct (PAC)-Bayes framework to obtain an…