Related papers: Convergence Rates and Decoupling in Linear Stochas…

In a real Hilbert space $\mathcal{H}$. Given any function $f$ convex differentiable whose solution set $\argmin_{\mathcal{H}}\,f$ is nonempty, by considering the Proximal Algorithm $x_{k+1}=\text{prox}_{\b_k f}(d x_k)$, where $0<d<1$ and…

Fix $c\in (0,1)$ and let $\Gamma$ be a $\lfloor c n\rfloor$-regular digraph on $n$ vertices drawn uniformly at random. We prove that when $n$ is large, the (non-symmetric) adjacency matrix $M$ of $\Gamma$ is invertible with high…

Nearest neighbor is a popular class of classification methods with many desirable properties. For a large data set which cannot be loaded into the memory of a single machine due to computation, communication, privacy, or ownership…

The Kaczmarz method for solving linear systems of equations is an iterative algorithm that has found many applications ranging from computer tomography to digital signal processing. Despite the popularity of this method, useful theoretical…

A randomized Kaczmarz method was recently proposed for phase retrieval, which has been shown numerically to exhibit empirical performance over other state-of-the-art phase retrieval algorithms both in terms of the sampling complexity and in…

The vast majority of convergence rates analysis for stochastic gradient methods in the literature focus on convergence in expectation, whereas trajectory-wise almost sure convergence is clearly important to ensure that any instantiation of…

Suppose a matrix $A \in \mathbb{R}^{m \times n}$ of rank $r$ with singular value decomposition $A = U_{A}\Sigma_{A} V_{A}^{T}$, where $U_{A} \in \mathbb{R}^{m \times r}$, $V_{A} \in \mathbb{R}^{n \times r}$ are orthonormal and $\Sigma_{A}…

We propose a new type of approximate counting algorithms for the problems of enumerating the number of independent sets and proper colorings in low degree graphs with large girth. Our algorithms are not based on a commonly used Markov chain…

For any finite point set in $D$-dimensional space equipped with the 1-norm, we present random linear embeddings to $k$-dimensional space, with a new metric, having the following properties. For any pair of points from the point set that are…

Estimation of actual errors from the residue in iterative solutions is necessary for efficient solution of large problems when their condition number is much larger than one. Such estimators for conjugate gradient algorithms used to solve…

Approximate message passing (AMP) methods and their variants have attracted considerable recent attention for the problem of estimating a random vector $\mathbf{x}$ observed through a linear transform $\mathbf{A}$. In the case of large…

For a broad class of models widely used in practice for choice and ranking data based on Luce's choice axiom, including the Bradley--Terry--Luce and Plackett--Luce models, we show that the associated maximum likelihood estimation problems…

Motivated by metastability in the zero-range process, we consider i.i.d.\ random variables with values in $\N_0$ and Weibull-like (stretched exponential) law $\mathbb P(X_i =k) = c \exp( - k^\alpha)$, $\alpha \in (0,1)$. We condition on…

A positive linear recurrence sequence is of the form $H_{n+1} = c_1 H_n + \cdots + c_L H_{n+1-L}$ with each $c_i \ge 0$ and $c_1 c_L > 0$, with appropriately chosen initial conditions. There is a notion of a legal decomposition (roughly,…

We present a randomized algorithm which takes as input an undirected graph $G$ on $n$ vertices with maximum degree $\Delta$, and a number of colors $k \geq (8/3 + o_{\Delta}(1))\Delta$, and returns -- in expected time…

Let $M_n$ be a random $n\times n$ matrix with i.i.d. $\text{Bernoulli}(1/2)$ entries. We show that for fixed $k\ge 1$, \[\lim_{n\to \infty}\frac{1}{n}\log_2\mathbb{P}[\text{corank }M_n\ge k] = -k.\]

We study largest singular values of large random matrices, each with mean of a fixed rank $K$. Our main result is a limit theorem as the number of rows and columns approach infinity, while their ratio approaches a positive constant. It…

The Kaczmarz method is a popular iterative scheme for solving large-scale linear systems. The randomized Kaczmarz method (RK) greatly improves the convergence rate of the Kaczmarz method, by using the rows of the coefficient matrix in…

We study randomized variants of two classical algorithms: coordinate descent for systems of linear equations and iterated projections for systems of linear inequalities. Expanding on a recent randomized iterated projection algorithm of…

Motivated by applications in reinforcement learning (RL), we study a nonlinear stochastic approximation (SA) algorithm under Markovian noise, and establish its finite-sample convergence bounds under various stepsizes. Specifically, we show…