Related papers: PAC-Bayesian Inequalities for Martingales
A central challenge in understanding generalization is to obtain non-vacuous guarantees that go beyond worst-case complexity over data or weight space. Among existing approaches, PAC-Bayes bounds stand out as they can provide tight,…
We consider the problem of bounding large deviations for non-i.i.d. random variables that are allowed to have arbitrary dependencies. Previous works typically assumed a specific dependence structure, namely the existence of independent…
We investigate the conditional distributions of two Banach space valued, jointly Gaussian random variables. In particular, we show that these conditional distributions are again Gaussian and that their means and covariances can be…
We give the cumulative distribution functions, the expected values, and the moments of weighted lattice polynomials when regarded as real functions of independent random variables. Since weighted lattice polynomial functions include…
Covariate balancing is a popular technique for controlling confounding in observational studies. It finds weights for the treatment group which are close to uniform, but make the group's covariate means (approximately) equal to those of the…
Since its original formulation, Jensen's inequality has played a fundamental role across mathematics, statistics, and machine learning, with its probabilistic version highlighting the nonnegativity of the so-called Jensen's gap, i.e., the…
The concept of conditional expectation is important in applications of probability and statistics in many areas such as reliability engineering, economy, finance, and actuarial sciences due to its property of being the best predictor of a…
The paper is devoted to establishing some general exponential inequalities for supermartingales. The inequalities improve or generalize many exponential inequalities of Bennett, Freedman, de la Pe\~{n}a, Pinelis and van de Geer. Moreover,…
Datasets displaying temporal dependencies abound in science and engineering applications, with Markov models representing a simplified and popular view of the temporal dependence structure. In this paper, we consider Bayesian settings that…
Variational approximation techniques and inference for stochastic models in machine learning has gained much attention the last years. Especially in the case of Gaussian Processes (GP) and their deep versions, Deep Gaussian Processes…
In statistical learning theory, a generalization bound usually involves a complexity measure imposed by the considered theoretical framework. This limits the scope of such bounds, as other forms of capacity measures or regularizations are…
This note extends some results of Nishiyama [Ann. Probab. 28 (2000) 685--712]. A maximal inequality for stochastic integrals with respect to integer-valued random measures which may have infinitely many jumps on compact time intervals is…
The push-forward operation enables one to redistribute a probability measure through a deterministic map. It plays a key role in statistics and optimization: many learning problems (notably from optimal transport, generative modeling, and…
In recent years, stochastic dominance for independent and identically distributed (iid) infinite-mean random variables has received considerable attention. The literature has identified several classes of distributions of nonnegative random…
Learning sentence vectors from an unlabeled corpus has attracted attention because such vectors can represent sentences in a lower dimensional and continuous space. Simple heuristics using pre-trained word vectors are widely applied to…
This paper presents eight PAC-Bayes bounds to analyze the generalization performance of multi-view classifiers. These bounds adopt data dependent Gaussian priors which emphasize classifiers with high view agreements. The center of the prior…
Class imbalance poses a significant challenge in classification tasks, where traditional approaches often lead to biased models and unreliable predictions. Undersampling and oversampling techniques have been commonly employed to address…
We extend Fano's inequality, which controls the average probability of events in terms of the average of some $f$--divergences, to work with arbitrary events (not necessarily forming a partition) and even with arbitrary $[0,1]$--valued…
We extend some sharp inequalities for martingale-differences to general multiplicative systems of random variables. The key ingredient in the proofs is a technique reducing the general case to the case of Rademacher random variables without…
Group-invariant probability distributions appear in many data-generative models in machine learning, such as graphs, point clouds, and images. In practice, one often needs to estimate divergences between such distributions. In this work, we…