Related papers: Functional Central Limit Theorem for Stochastic Gr…
We identity the optimal non-infinitesimal direction of descent for a convex function. An algorithm is developed that can theoretically minimize a subset of (non-convex) functions.
An usual problem in statistics consists in estimating the minimizer of a convex function. When we have to deal with large samples taking values in high dimensional spaces, stochastic gradient algorithms and their averaged versions are…
The approximation of integral type functionals is studied for discrete observations of a continuous It\^o semimartingale. Based on novel approximations in the Fourier domain, central limit theorems are proved for $L^2$-Sobolev functions…
In this work, we obtain the central limit theorem for fluctuations of Young diagrams around their limit shape in the bulk of the "spectrum" of partitions of a large integer n (under the Plancherel measure). More specifically, we show that,…
In this article, we quantify the functional convergence of the rescaled random walk with heavy tails to a stable process.This generalizes the Generalized Central Limit Theorem for stable random variables infinite dimension. We show that…
We study the statistical inference of nonlinear stochastic approximation algorithms utilizing a single trajectory of Markovian data. Our methodology has practical applications in various scenarios, such as Stochastic Gradient Descent (SGD)…
We prove the local convergence to minima and estimates on the rate of convergence for the stochastic gradient descent method in the case of not necessarily globally convex nor contracting objective functions. In particular, the results are…
We revisit the classical dual ascent algorithm for minimization of convex functionals in the presence of linear constraints, and give convergence results which apply even for non-convex functionals. We describe limit points in terms of the…
Motivated by the extensive application of approximate gradients in machine learning and optimization, we investigate inexact subgradient methods subject to persistent additive errors. Within a nonconvex semialgebraic framework, assuming…
Statistical machine learning models trained with stochastic gradient algorithms are increasingly being deployed in critical scientific applications. However, computing the stochastic gradient in several such applications is highly expensive…
In this paper we analyze the behaviour of the stochastic gradient descent (SGD), a widely used method in supervised learning for optimizing neural network weights via a minimization of non-convex loss functions. Since the pioneering work of…
We consider the problem of minimizing a convex function that is evolving according to unknown and possibly stochastic dynamics, which may depend jointly on time and on the decision variable itself. Such problems abound in the machine…
In this paper we propose a distributed version of a randomized block-coordinate descent method for minimizing the sum of a partially separable smooth convex function and a fully separable non-smooth convex function. Under the assumption of…
We study inhomogeneous random graphs with a finite type space. For a natural generalization of the model as a dynamic network-valued process, the paper establishes the following results: (a) Functional central limit theorems for the…
The convergence of stochastic interacting particle systems in the mean-field limit to solutions of conservative stochastic partial differential equations is established, with optimal rate of convergence. As a second main result, a…
The main result of this paper is a general central limit theorem for distributions defined by certain renewal type equations. We apply this to weakly self-avoiding random walks. We give good error estimates and Gaussian tail estimates which…
This article suggests that deterministic Gradient Descent, which does not use any stochastic gradient approximation, can still exhibit stochastic behaviors. In particular, it shows that if the objective function exhibit multiscale…
The purpose of the present paper is to establish explicit bounds on moderate deviation probabilities for a rather general class of geometric functionals enjoying the stabilization property, under Poisson input and the assumption of a…
This paper establishes central limit theorems for Polyak-Ruppert averaged Q-learning under asynchronous updates. We prove a non-asymptotic central limit theorem, where the convergence rate in Wasserstein distance explicitly reflects the…
A variant of consensus based distributed gradient descent (\textbf{DGD}) is studied for finite sums of smooth but possibly non-convex functions. In particular, the local gradient term in the fixed step-size iteration of each agent is…