Related papers: Tight Bounds for Jensen's Gap with Applications to…
We present here a PAC-Bayesian point of view on adaptive supervised classification. Using convex analysis, we show how to get local measures of the complexity of the classification model involving the relative entropy of posterior…
Variational inference is a general approach for approximating complex density functions, such as those arising in latent variable models, popular in machine learning. It has been applied to approximate the maximum likelihood estimator and…
To answer questions of "causes of effects", the probability of necessity is introduced for assessing whether or not an observed outcome was caused by an earlier treatment. However, the statistical inference for probability of necessity is…
We present an extensive analysis of relative deviation bounds, including detailed proofs of two-sided inequalities and their implications. We also give detailed proofs of two-sided generalization bounds that hold in the general case of…
Since the celebrated works of Russo and Zou (2016,2019) and Xu and Raginsky (2017), it has been well known that the generalization error of supervised learning algorithms can be bounded in terms of the mutual information between their input…
This paper develops upper and lower bounds for the probability of Boolean functions by treating multiple occurrences of variables as independent and assigning them new individual probabilities. We call this approach dissociation and give an…
We first introduce the class of strictly quasiconvex and strictly quasiconcave Jensen divergences which are oriented (asymmetric) distances, and study some of their properties. We then define the strictly quasiconvex Bregman divergences as…
We present a unified technique for sequential estimation of convex divergences between distributions, including integral probability metrics like the kernel maximum mean discrepancy, $\varphi$-divergences like the Kullback-Leibler…
The ultimate performance of machine learning algorithms for classification tasks is usually measured in terms of the empirical error probability (or accuracy) based on a testing dataset. Whereas, these algorithms are optimized through the…
A loss function measures the discrepancy between the true values and their estimated fits, for a given instance of data. In classification problems, a loss function is said to be proper if a minimizer of the expected loss is the true…
Mapping data from and/or onto a known family of distributions has become an important topic in machine learning and data analysis. Deep generative models (e.g., generative adversarial networks ) have been used effectively to match known and…
A new risk bound is presented for the problem of convex/concave function estimation, using the least squares estimator. The best known risk bound, as had appeared in \citet{GSvex}, scaled like $\log(en) n^{-4/5}$ under the mean squared…
The push-forward operation enables one to redistribute a probability measure through a deterministic map. It plays a key role in statistics and optimization: many learning problems (notably from optimal transport, generative modeling, and…
We consider the variance of a function of $n$ independent random variables and provide new inequalities which, in particular, extend previous results obtained for symmetric functions in the i.i.d.~setting. For instance, we obtain various…
We propose a family of variational approximations to Bayesian posterior distributions, called $\alpha$-VB, with provable statistical guarantees. The standard variational approximation is a special case of $\alpha$-VB with $\alpha=1$. When…
Coherent lower previsions are general probabilistic models allowing incompletely specified probability distributions. However, for complete description of a coherent lower prevision -- even on finite underlying sample spaces -- an infinite…
We investigate quantitative implications of the notion of log-concavity through a probabilistic interpretation. In particular, we derive concentration inequalities, moment and entropy bounds for random variables satisfying a precise degree…
We give a variational formulation for $-\log\mathbb{E}_\nu\left[e^{-f}|\mathcal{F}_t\right]$ for a large class of measures $\nu$. We give a refined entropic characterization of the invertibility of some perturbations of the identity. We…
Some of the important inequalities associated with quantum entropy are immediate algebraic consequences of the Hansen-Pedersen-Jensen inequality. A general argument is given using matrix perspectives of operator convex functions. A matrix…
In this paper, we consider the sublinear expectation on bounded random variables. With the notion of uncorrelatedness for random variables under the sublinear expectation, a weak law of large numbers is obtained. With the notion of…