Related papers: Central limit theorems for stochastic gradient des…
We consider the least-squares regression problem and provide a detailed asymptotic analysis of the performance of averaged constant-step-size stochastic gradient descent (a.k.a. least-mean-squares). In the strongly-convex case, we provide…
We establish some limit theorems for one-dimensional elephant random walk, including Berry-Esseen bounds, Cram\'{e}r moderate deviations and local limit theorems. These limit theorems can be regarded as refinements of the central limit…
The strong growth condition (SGC) is known to be a sufficient condition for linear convergence of the stochastic gradient method using a constant step-size $\gamma$ (SGM-CS). In this paper, we provide a necessary condition, for the linear…
We prove a centre manifold theorem for a map along a manifold-with-boundary of fixed points, and provide an application to the study of gradient descent with large step size on two-layer matrix factorisation problems.
In this article we review recent generalisations of the central limit theorem for the sum of specially correlated (or q-independent) variables, focusing on q greater or equal than 1. Specifically, this kind of correlation turns the…
The purpose of the present paper is to establish explicit bounds on moderate deviation probabilities for a rather general class of geometric functionals enjoying the stabilization property, under Poisson input and the assumption of a…
We investigate symmetric edge polytopes generated by Erd\H{o}s--R\'enyi random graphs in a high-dimensional regime. These objects provide a natural and largely unexplored model of random lattice polytopes, in which geometric properties are…
The {\L}ojasiewicz inequality characterizes objective-value convergence along gradient flows and, in special cases, yields exponential decay of the cost. However, such results do not directly give rates of convergence in the state. In this…
Although it is relatively easy to apply, the gradient method often displays a disappointingly slow rate of convergence. Its convergence is specially based on the structure of the matrix of the algebraic linear system, and on the choice of…
Gradient normalization and soft clipping are two popular techniques for tackling instability issues and improving convergence of stochastic gradient descent (SGD) with momentum. In this article, we study these types of methods through the…
We consider a slow-fast stochastic differential system with L\'evy noise. We will employ the perturbed test function method to study the normal deviation of the slow-fast system. Our main result states that the deviation can be approximated…
Stochastic gradient descent with momentum (SGDM) is one of the most widely used optimization algorithms in machine learning. While optimization properties of SGDM have been extensively studied in the literature, it remains insufficiently…
We provide rates of convergence in the central limit theorem in terms of projective criteria for adapted stationary sequences of centered random variables taking values in Banach spaces, with finite moment of order $p \in ]2,3]$ as soon as…
In this paper, we study a class of unbalanced step-reinforced random walks that unifies the elephant random walk, the positively step-reinforced random walk, and the negatively step-reinforced random walk. By establishing a connection with…
We analyze the fluctuations of incomplete $U$-statistics over a triangular array of independent random variables. We give criteria for a Central Limit Theorem (CLT, for short) to hold in the sense that we prove that an appropriately scaled…
Variation of empirical Fr\'echet means on a metric space with curvature bounded above is encoded via random fields indexed by unit tangent vectors. A central limit theorem shows these random tangent fields converge to a Gaussian such field…
We solve a regularized weighted low-rank approximation problem by a stochastic gradient descent on a manifold. To guarantee the convergence of our stochastic gradient descent, we establish a convergence theorem on manifolds for…
We prove a central limit theorem under diffusive scaling for the displacement of a random walk on ${\mathbb Z}^d$ in stationary and ergodic doubly stochastic random environment, under the $\mathcal{H}_{-1}$-condition imposed on the drift…
We study conditions under which treatment effect estimators constructed under the no-interference assumption in randomized experiments are asymptotically normal in the presence of interference. We prove that the standard Horvitz-Thompson…
We established the rate of convergence in the central limit theorem for stopped sums of a class of martingale difference sequences.