Related papers: The conjugate gradient algorithm on a general clas…
For a class of symmetric random matrices whose entries are martingale differences adapted to an increasing filtration, we prove that under a Lindeberg-like condition, the empirical spectral distribution behaves asymptotically similarly to a…
The stochastic gradient descent (SGD) optimization algorithm plays a central role in a series of machine learning applications. The scientific literature provides a vast amount of upper error bounds for the SGD method. Much less attention…
We consider a class of random quantum circuits where at each step a gate from a universal set is applied to a random pair of qubits, and determine how quickly averages of arbitrary finite-degree polynomials in the matrix elements of the…
Online averaged stochastic gradient algorithms are more and more studied since (i) they can deal quickly with large sample taking values in high dimensional spaces, (ii) they enable to treat data sequentially, (iii) they are known to be…
The conjugate gradient method (CG) is typically used with a preconditioner which improves efficiency and robustness of the method. Many preconditioners include parameters and a proper choice of a preconditioner and its parameters is often…
The asymptotic behavior of stochastic gradient algorithms is studied. Relying on results from differential geometry (Lojasiewicz gradient inequality), the single limit-point convergence of the algorithm iterates is demonstrated and…
Our goal in this paper is to clarify the relationship between the block Lanczos and the block conjugate gradient (BCG) algorithms. Under the full rank assumption for the block vectors, we show the one-to-one correspondence between the…
We study the averaging-based distributed optimization solvers over random networks. We show a general result on the convergence of such schemes using weight-matrices that are row-stochastic almost surely and column-stochastic in expectation…
We propose a stochastic gradient framework for solving stochastic composite convex optimization problems with (possibly) infinite number of linear inclusion constraints that need to be satisfied almost surely. We use smoothing and homotopy…
The adjoint method is an efficient way to numerically compute gradients in optimization problems with constraints, but is only formulated to differentiable cost and constraint functions on real variables. With the introduction of complex…
We consider large Hermitian matrices whose entries are defined by evaluating the exponential function along orbits of the skew-shift $\binom{j}{2} \omega+jy+x \mod 1$ for irrational $\omega$. We prove that the eigenvalue distribution of…
In this paper, we study convergence properties of the gradient Expectation-Maximization algorithm \cite{lange1995gradient} for Gaussian Mixture Models for general number of clusters and mixing coefficients. We derive the convergence rate…
The gradient mapping norm is a strong and easily verifiable stopping criterion for first-order methods on composite problems. When the objective exhibits the quadratic growth property, the gradient mapping norm minimization problem can be…
We propose a general error analysis related to the low-rank approximation of a given real matrix in both the spectral and Frobenius norms. First, we derive deterministic error bounds that hold with some minimal assumptions. Second, we…
We study the iterative solution of linear systems of equations arising from stochastic Galerkin finite element discretizations of saddle point problems. We focus on the Stokes model with random data parametrized by uniformly distributed…
Gradient boosting of prediction rules is an efficient approach to learn potentially interpretable yet accurate probabilistic models. However, actual interpretability requires to limit the number and size of the generated rules, and existing…
Multicalibration gradient boosting has recently emerged as a scalable method that empirically produces approximately multicalibrated predictors and has been deployed at web scale. Despite this empirical success, its convergence properties…
We extend the proof of the local semicircle law for generalized Wigner matrices given in [4] to the case when the matrix of variances has an eigenvalue $ -1 $. In particular, this result provides a short proof of the optimal local…
We compute averages of products and ratios of characteristic polynomials associated with Orthogonal, Unitary, and Symplectic Ensembles of Random Matrix Theory. The pfaffian/determinantal formulas for these averages are obtained, and the…
We consider decentralized machine learning over a network where the training data is distributed across $n$ agents, each of which can compute stochastic model updates on their local data. The agent's common goal is to find a model that…