Related papers: Central limit theorems for stochastic gradient des…

This paper studies fixed step-size stochastic approximation (SA) schemes, including stochastic gradient schemes, in a Riemannian framework. It is motivated by several applications, where geodesics can be computed explicitly, and their use…

We consider the rates of convergence of the quenched central limit theorem for hitting times of one-dimensional random walks in a random environment. Previous results had identified polynomial upper bounds for the rates of decay which are…

The Central Limit Theorem (CLT) establishes that sufficiently large sequences of independent and identically distributed random variables converge in probability to a normal distribution. This makes the CLT a fundamental building block of…

We prove a central limit theorem for the components of the largest eigenvectors of the adjacency matrix of a finite-dimensional random dot product graph whose true latent positions are unknown. In particular, we follow the methodology…

Based on a result by Taylor, Hendrickx, and Glineur (J. Optim. Theory Appl., 178(2):455--476, 2018) on the attainable convergence rate of gradient descent for smooth and strongly convex functions in terms of function values, an elementary…

The statistical scattering properties of wave transport in disordered waveguides are derived perturbatively within the transition matrix formalism. The limiting macroscopic statistic of the wave transport, emerges as a consequence of a…

We give a new characterization for the convergence in distribution to a standard normal law of a sequence of multiple stochastic integrals of a fixed order with variance one, in terms of the Malliavin derivatives of the sequence. We extend…

We study the convergence issue for the gradient algorithm (employing general step sizes) for optimization problems on general Riemannian manifolds (without curvature constraints). Under the assumption of the local convexity/quasi-convexity…

We study the generalization error of randomized learning algorithms -- focusing on stochastic gradient descent (SGD) -- using a novel combination of PAC-Bayes and algorithmic stability. Importantly, our generalization bounds hold for all…

We study gradient descent (GD) with a constant stepsize for $\ell_2$-regularized logistic regression with linearly separable data. Classical theory suggests small stepsizes to ensure monotonic reduction of the optimization objective,…

Stochastic Gradient Descent (SGD) is widely used in machine learning research. Previous convergence analyses of SGD under the vanishing step-size setting typically require Robbins-Monro conditions. However, in practice, a wider variety of…

We develop the theory of discrete-time gradient flows for convex functions on Alexandrov spaces with arbitrary upper or lower curvature bounds. We employ different resolvent maps in the upper and lower curvature bound cases to construct…

We consider a recursive algorithm to construct an aggregated estimator from a finite number of base decision rules in the classification problem. The estimator approximately minimizes a convex risk functional under the l1-constraint. It is…

Consider generalized adapted stochastic integrals with respect to independently scattered random measures with second moments. We use a decoupling technique, known as the "principle of conditioning", to study their stable convergence…

The present paper deals with autonomous integral equations with infinite delay via dynamical system approach. Existence, local exponential attractivity, and other properties of center manifold are established by means of the…

We derive a second-order ordinary differential equation (ODE) which is the limit of Nesterov's accelerated gradient method. This ODE exhibits approximate equivalence to Nesterov's scheme and thus can serve as a tool for analysis. We show…

In this paper, we prove a central limit theorem for a sequence of iterated Shorohod integrals using the techniques of Malliavin calculus. The convergence is stable, and the limit is a conditionally Gaussian random variable. Some…

We consider stochastic gradient descent algorithms for minimizing a non-smooth, strongly-convex function. Several forms of this algorithm, including suffix averaging, are known to achieve the optimal $O(1/T)$ convergence rate in…

When implementing the gradient descent method in low precision, the employment of stochastic rounding schemes helps to prevent stagnation of convergence caused by the vanishing gradient effect. Unbiased stochastic rounding yields zero bias…

This paper derives central limit and bootstrap theorems for probabilities that sums of centered high-dimensional random vectors hit hyperrectangles and sparsely convex sets. Specifically, we derive Gaussian and bootstrap approximations for…