Related papers: PAC-Bayes Iterated Logarithm Bounds for Martingale…

Classical PAC generalization bounds on the prediction risk of a classifier are insufficient to provide theoretical guarantees on fairness when the goal is to learn models balancing predictive risk and fairness constraints. We propose a…

Bayesian priors offer a compact yet general means of incorporating domain knowledge into many learning tasks. The correctness of the Bayesian analysis and inference, however, largely depends on accuracy and correctness of these priors.…

Stochastic Network Calculus is a probabilistic method to compute performance bounds in networks, such as end-to-end delays. It relies on the analysis of stochastic processes using formalism of (Deterministic) Network Calculus. However,…

In this note, we improve some concentration inequalities for martingales with bounded increments. These results recover the missing factor in Freedman-style inequalities and are near optimal. We also provide minor refinements of…

We use the abstract method of (local) martingale problems in order to give criteria for convergence of stochastic processes. Extending previous notions, the formulation we use is neither restricted to Markov processes (or semimartingales),…

This tutorial gives a concise overview of existing PAC-Bayesian theory focusing on three generalization bounds. The first is an Occam bound which handles rules with finite precision parameters and which states that generalization loss is…

New Vapnik and Chervonenkis type concentration inequalities are derived for the empirical distribution of an independent random sample. Focus is on the maximal deviation over classes of Borel sets within a low probability region. The…

The developments of Rademacher complexity and PAC-Bayesian theory have been largely independent. One exception is the PAC-Bayes theorem of Kakade, Sridharan, and Tewari (2008), which is established via Rademacher complexity theory by…

Let $(\Omega, \A, \mu)$ be a Lebesgue space and $T$ an ergodic measure preserving automorphism on $\Omega$ with positive entropy. We show that there is a bounded and strictly stationary martingale difference sequence defined on $\Omega$…

We provide new upper bounds for mixing times of general finite Markov chains. We use these bounds to show that the total variation mixing time is robust under rough isometry for bounded degree graphs that are roughly isometric to trees.

Most PAC-Bayesian bounds hold in the batch learning setting where data is collected at once, prior to inference or prediction. This somewhat departs from many contemporary learning problems where data streams are collected and the…

We consider Markov chains which are polynomially mixing, in a weak sense expressed in terms of the space of functions on which the mixing speed is controlled. In this context, we prove polynomial large and moderate deviations inequalities.…

We construct a class of nonnegative martingale processes that oscillate indefinitely with high probability. For these processes, we state a uniform rate of the number of oscillations and show that this rate is asymptotically close to the…

We use martingales to study Bayesian consistency. We derive sufficient conditions for both Hellinger and Kullback-Leibler consistency, which do not rely on the use of a sieve. Alternative sufficient conditions for Hellinger consistency are…

Some general connections between martingales and character ratios of finite groups are developed. As an application we sharpen the convergence rate in a central limit theorem for the character ratio of a random representation of the…

Say that a finite group $G$ is mixable if a product of random elements, each chosen independently from two options, can distribute uniformly on $G$. We present conditions and obstructions to mixability. We show that $2$-groups, the…

We give sufficient conditions for the bounded law of the iterated logarithms for strictly stationary random fields when the summation is done on rectangle. The study is done by the control of an appropriated maximal function. The case of…

Generalization in deep learning has been the topic of much recent theoretical and empirical research. Here we introduce desiderata for techniques that predict generalization errors for deep learning models in supervised learning. Such…

We present a new second-order oracle bound for the expected risk of a weighted majority vote. The bound is based on a novel parametric form of the Chebyshev- Cantelli inequality (a.k.a. one-sided Chebyshev's), which is amenable to efficient…

In this paper we derive a PAC-Bayesian error bound for autonomous stochastic LTI state-space models. The motivation for deriving such error bounds is that they will allow deriving similar error bounds for more general dynamical systems,…