Related papers: An Inequality with Applications to Structured Spar…
In this report, we aim to exemplify concentration inequalities and provide easy to understand proofs for it. Our focus is on the inequalities which are helpful in the design and analysis of machine learning algorithms.
Class imbalance remains a major challenge in machine learning, especially in multi-class problems with long-tailed distributions. Existing methods, such as data resampling, cost-sensitive techniques, and logistic loss modifications, though…
The present work provides an original framework for random matrix analysis based on revisiting the concentration of measure theory from a probabilistic point of view. By providing various notions of vector concentration ($q$-exponential,…
Sparsity-based models and techniques have been exploited in many signal processing and imaging applications. Data-driven methods based on dictionary and sparsifying transform learning enable learning rich image features from data, and can…
As the context window expands, self-attention increasingly dominates the transformer's inference time. Therefore, accelerating attention computation while minimizing performance degradation is essential for the efficient deployment of Large…
We prove an elementary yet useful inequality bounding the maximal value of certain linear programs. This leads directly to a bound on the martingale difference for arbitrarily dependent random variables, providing a generalization of some…
We study concentration inequalities for structured weighted sums of random data, including (i) tensor inner products and (ii) sequential matrix sums. We are interested in tail bounds and concentration inequalities for those structured…
Known sparsity thresholds for basis pursuit to deliver the maximally sparse solution of the compressed sensing recovery problem typically depend on the dictionary's coherence. While the coherence is easy to compute, it can lead to rather…
The classical Gaussian concentration inequality for Lipschitz functions is adapted to a setting where the classical assumptions (i.e. Lipschitz and Gaussian) are not met. The theory is more direct than much of the existing theory designed…
Matrix concentration inequalities provide information about the probability that a random matrix is close to its expectation with respect to the $l_2$ operator norm. This paper uses semigroup methods to derive sharp nonlinear matrix…
We consider the problem of aggregating the elements of a possibly infinite dictionary for building a decision procedure that aims at minimizing a given criterion. Along with the dictionary, an independent identically distributed training…
The dictionary learning problem concerns the task of representing data as sparse linear sums drawn from a smaller collection of basic building blocks. In application domains where such techniques are deployed, we frequently encounter…
Inspired by Kalikow-type decompositions, we introduce a new stochastic model of infinite neuronal networks, for which we establish sharp oracle inequalities for Lasso methods and restricted eigenvalue properties for the associated Gram…
As machine learning applications grow increasingly ubiquitous and complex, they face an increasing set of requirements beyond accuracy. The prevalent approach to handle this challenge is to aggregate a weighted combination of requirement…
We consider the stochastic integrals of multivariate point processes and study their concentration phenomena. In particular, we obtain a Bernstein type of concentration inequality through Dol\'eans-Dade exponential formula and a uniform…
We propose new concentration inequalities for self-normalized martingales. The main idea is to introduce a suitable weighted sum of the predictable quadratic variation and the total quadratic variation of the martingale. It offers much more…
In recent years, sparse principal component analysis has emerged as an extremely popular dimension reduction technique for high-dimensional data. The theoretical challenge, in the simplest case, is to estimate the leading eigenvector of a…
A dictionary is a database of standard vectors, so that other vectors / signals are expressed as linear combinations of dictionary vectors, and the task of learning a dictionary for a given data is to find a good dictionary so that the…
This study is about inducing classifiers using data that is imbalanced, with a minority class being under-represented in relation to the majority classes. The first section of this research focuses on the main characteristics of data that…
In this paper, we derive a new version of Hanson-Wright inequality for a sparse bilinear form of sub-Gaussian variables. Our results are generalization of previous deviation inequalities that consider either sparse quadratic forms or dense…