Related papers: Sharp concentration of uniform generalization erro…
We consider multi-label prediction problems with large output spaces under the assumption of output sparsity -- that the target (label) vectors have small support. We develop a general theory for a variant of the popular error correcting…
A problem of bounding the generalization error of a classifier f in H, where H is a "base" class of functions (classifiers), is considered. This problem frequently occurs in computer learning, where efficient algorithms of combining simple…
Learning an appropriate (dis)similarity function from the available data is a central problem in machine learning, since the success of many machine learning algorithms critically depends on the choice of a similarity function to compare…
We provide a novel dimension-free uniform concentration bound for the empirical risk function of constrained logistic regression. Our bound yields a milder sufficient condition for a uniform law of large numbers than conditions derived by…
Consider a proper geodesic metric space $(X,d)$ equipped with a Borel measure $\mu.$ We establish a family of uniform Poincar\'e inequalities on $(X,d,\mu)$ if it satisfies a local Poincar\'e inequality ($P_{loc}$) and a condition on growth…
We study the behavior of linear discriminant functions for binary classification in the infinite-imbalance limit, where the sample size of one class grows without bound while the sample size of the other remains fixed. The coefficients of…
We develop a unified Data Processing Inequality PAC-Bayesian framework -- abbreviated DPI-PAC-Bayesian -- for deriving generalization error bounds in the supervised learning setting. By embedding the Data Processing Inequality (DPI) into…
Lower and upper bounds are explored for the uniform (Kolmogorov) and $L^2$-distances between the distributions of weighted sums of dependent summands and the normal law. The results are illustrated for several classes of random variables…
We consider large uniform labeled random graphs in different classes with prescribed decorations in their modular decomposition. Our main result is the estimation of the number of copies of every graph as an induced subgraph. As a…
In the first part we study deviation of a polynomial from its mathematical expectation. This deviation can be estimated from above by Carbery--Wright inequality, so we investigate estimates of the deviation from below. We obtain such…
Being able to reliably assess not only the \emph{accuracy} but also the \emph{uncertainty} of models' predictions is an important endeavour in modern machine learning. Even if the model generating the data and labels is known, computing the…
In this article, we consider scenarios in which traditional estimates for the active subspace method based on probabilistic Poincar\'e inequalities are not valid due to unbounded Poincar\'e constants. Consequently, we propose a framework…
We consider the classification problem of a high-dimensional mixture of two Gaussians with general covariance matrices. Using the replica method from statistical physics, we investigate the asymptotic behavior of a general class of…
We study concentration inequalities for the Kullback--Leibler (KL) divergence between the empirical distribution and the true distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes…
Emerging applications of sensor networks for detection sometimes suggest that classical problems ought be revisited under new assumptions. This is the case of binary hypothesis testing with independent - but not necessarily identically…
We derive sufficient conditions for a probability measure on a finite product space (a spin system) to satisfy a (modified) logarithmic Sobolev inequality. We establish these conditions for various examples, such as the (vertex-weighted)…
We prove new concentration estimates for random variables that are functionals of a Poisson measure defined on a general measure space. Our results are specifically adapted to geometric applications, and are based on a pervasive use of a…
Group-invariant probability distributions appear in many data-generative models in machine learning, such as graphs, point clouds, and images. In practice, one often needs to estimate divergences between such distributions. In this work, we…
We observe $(X_i,Y_i)_{i=1}^n$ where the $Y_i$'s are real valued outputs and the $X_i$'s are $m\times T$ matrices. We observe a new entry $X$ and we want to predict the output $Y$ associated with it. We focus on the high-dimensional…
We derive novel concentration inequalities that bound the statistical error for a large class of stochastic optimization problems, focusing on the case of unbounded objective functions. Our derivations utilize the following key tools: 1) A…