Related papers: On Concentration Inequalities for Random Matrix Pr…

Trace norm regularization is a widely used approach for learning low rank matrices. A standard optimization strategy is based on formulating the problem as one of low rank matrix factorization which, however, leads to a non-convex problem.…

We establish two concentration inequalities for nonlinear stochastic system under time-varying contraction conditions. The key to our approach is an energy function termed Averaged Moment Generating Function (AMGF). By combining it with…

We study deterministic and stochastic primal-dual sub-gradient algorithms for distributed optimization of a separable objective function with global inequality constraints. In both algorithms, the norm of the Lagrangian multipliers are…

We analyse the matrix factorization problem. Given a noisy measurement of a product of two matrices, the problem is to estimate back the original matrices. It arises in many applications such as dictionary learning, blind matrix…

Let $\{A_{i,n}\}$ be a triangular array of elements in a Banach algebra, whose norms do not grow too fast, and whose row averages converge to $A$. Let $\sigma \in S(n)$ be a permutation drawn uniformly at random. If the array only contains…

At the core of the popular Transformer architecture is the self-attention mechanism, which dynamically assigns softmax weights to each input token so that the model can focus on the most salient information. However, the softmax structure…

Let $P_n$ be the $n$-step right product $A_1\cdots A_n$, where $A_1,A_2,\dots$ is a given infinite sequence of $d\times d$ matrices with nonnegative entries. In a wide range of situations, the normalized matrix product $P_n/{\Vert…

We prove that a sum of random matrices generated by a $\psi$-mixing Markov chain has similar spectral properties to a Gaussian matrix with the same mean and covariance structure. This nonasymptotic universality principle enables sharp…

In this work, we address the problem of sensor selection for state estimation via Kalman filtering. We consider a linear time-invariant (LTI) dynamical system subject to process and measurement noise, where the sensors we use to perform…

Data often comes in the form of an array or matrix. Matrix factorization techniques attempt to recover missing or corrupted entries by assuming that the matrix can be written as the product of two low-rank matrices. In other words, matrix…

We study the averaging-based distributed optimization solvers over random networks. We show a general result on the convergence of such schemes using weight-matrices that are row-stochastic almost surely and column-stochastic in expectation…

Matching is a widely used causal inference design that aims to approximate a randomized experiment using observational data by forming matched sets of treated and control units based on similarities in their covariates. Ideally, treated…

Random matrix theory has played an important role in recent work on statistical network analysis. In this paper, we review recent results on regimes of concentration of random graphs around their expectation, showing that dense graphs…

Inspired by a recent paper of I. Grama, E. Le Page and M. Peign\'e, we consider a sequence $(g_n)_{n \geq 1}$ of i.i.d. random $d\times d$-matrices with non-negative entries and study the fluctuations of the process $(\log \vert g_n\cdots…

We give two algorithms for output-sparse matrix multiplication (OSMM), the problem of multiplying two $n \times n$ matrices $A, B$ when their product $AB$ is promised to have at most $O(n^{\delta})$ many non-zero entries for a given value…

The matrix Markov inequality by Ahlswede was stated using the Loewner anti-order between positive definite matrices. Wang use this to derive several other Chebyshev and Chernoff-type inequalities (Hoeffding, Bernstein, empirical Bernstein)…

Using a martingale concentration inequality, concentration bounds `from time $n_0$ on' are derived for stochastic approximation algorithms with contractive maps and both martingale difference and Markov noises. These are applied to…

We consider the problem of convergence to zero of matrix products $A_{n}B_{n}\cdots A_{1}B_{1}$ with factors from two sets of matrices, $A_{i}\in\mathscr{A}$ and $B_{i}\in\mathscr{B}$, due to a suitable choice of matrices $\{B_{i}\}$. It is…

We revisit the well-studied problem of approximating a matrix product, $\mathbf{A}^T\mathbf{B}$, based on small space sketches $\mathcal{S}(\mathbf{A})$ and $\mathcal{S}(\mathbf{B})$ of $\mathbf{A} \in \R^{n \times d}$ and $\mathbf{B}\in…

Bayesian methods for low-rank matrix completion with noise have been shown to be very efficient computationally. While the behaviour of penalized minimization methods is well understood both from the theoretical and computational points of…