Related papers: An Inequality with Applications to Structured Spar…
In the machine learning era, sparsity continues to attract significant interest due to the benefits it provides to learning models. Algorithms aiming to optimise the \(\ell_0\)- and \(\ell_1\)-norm are the common choices to achieve…
We present a convex formulation of dictionary learning for sparse signal decomposition. Convexity is obtained by replacing the usual explicit upper bound on the dictionary size by a convex rank-reducing term similar to the trace norm. In…
We investigate geometric features of the unit ball corresponding to the sum of the nuclear norm of a matrix and the $l_1$ norm of its entries --- a common penalty function encouraging joint low rank and high sparsity. As a byproduct of this…
In this paper we derive tail bounds on the norms of random submatrices with non-uniformly distributed supports. We apply these results to sparse approximation and conduct an analysis of the average case performance of thresholding,…
The martingale method is used to establish concentration inequalities for a class of dependent random sequences on a countable state space, with the constants in the inequalities expressed in terms of certain mixing coefficients. Along the…
In a companion article we have introduced a notion of multiscale functional inequalities for functions $X(A)$ of an ergodic stationary random field $A$ on the ambient space $\mathbb R^d$. These inequalities are multiscale weighted versions…
We propose a novel combination of optimization tools with learning theory bounds in order to analyze the sample complexity of optimal kernel sum classifiers. This contrasts the typical learning theoretic results which hold for all…
The paper is devoted to establishing some general exponential inequalities for supermartingales. The inequalities improve or generalize many exponential inequalities of Bennett, Freedman, de la Pe\~{n}a, Pinelis and van de Geer. Moreover,…
As an alternative to the well-known methods of "chaining" and "bracketing" that have been developed in the study of random fields, a new method, which is based on a {\em stochastic maximal inequality} derived by using the formula for…
It is proved that the limiting distribution of the length of the longest weakly increasing subsequence in an inhomogeneous random word is related to the distribution function for the eigenvalues of a certain direct sum of Gaussian unitary…
In this paper we prove multilevel concentration inequalities for bounded functionals $f = f(X_1, \ldots, X_n)$ of random variables $X_1, \ldots, X_n$ that are either independent or satisfy certain logarithmic Sobolev inequalities. The…
We obtain the tail probability of generalized sub-Gaussian canonical processes. It can be viewed as a variant of the Bernstein-type inequality in the i.i.d case, and we further get a tighter bound of concentration inequality through…
We study supervised learning problems using clustering constraints to impose structure on either features or samples, seeking to help both prediction and interpretation. The problem of clustering features arises naturally in text…
When training data is sparse, more domain knowledge must be incorporated into the learning algorithm in order to reduce the effective size of the hypothesis space. This paper builds on previous work in which knowledge about qualitative…
Most classification models can be considered as the process of matching templates. However, when intra-class uncertainty/variability is not considered, especially for datasets containing unbalanced classes, this may lead to classification…
This note demonstrates that it is possible to bound the expectation of an arbitrary norm of a random matrix drawn from the Stiefel manifold in terms of the expected norm of a standard Gaussian matrix with the same dimensions. A related…
Audio events are quite often overlapping in nature, and more prone to noise than visual signals. There has been increasing evidence for the superior performance of representations learned using sparse dictionaries for applications like…
Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. While naturally cast as a combinatorial optimization problem, variable or feature selection admits a convex relaxation through the…
In this article we present a Bernstein inequality for sums of random variables which are defined on a spatial lattice structure. The inequality can be used to derive concentration inequalities. It can be useful to obtain consistency…
We give a distribution-dependent concentration inequality for functions of independent variables. The result extends Bernstein's inequality from sums to more general functions, whose variation in any argument does not depend too much on the…