Related papers: Functional Central Limit Theorem for Stochastic Gr…
Our work is part of the close link between continuous-time dissipative dynamical systems and optimization algorithms, and more precisely here, in the stochastic setting. We aim to study stochastic convex minimization problems through the…
A finite range interacting particle system on a transitive graph is considered. Assuming that the dynamics and the initial measure are invariant, the normalized empirical distribution process converges in distribution to a centered…
This paper proposes an asymptotic theory for online inference of the stochastic gradient descent (SGD) iterates with dropout regularization in linear regression. Specifically, we establish the geometric-moment contraction (GMC) for constant…
We prove convergence of a single time-scale stochastic subgradient method with subgradient averaging for constrained problems with a nonsmooth and nonconvex objective function having the property of generalized differentiability. As a tool…
Stochastic gradient descent (SGD) has been studied extensively over the past decades due to its simplicity and broad applicability in machine learning. In this work, we analyze the local behavior of gradient descent and stochastic gradient…
This paper studies a novel algorithm for nonconvex composite minimization which can be interpreted in terms of dual space nonlinear preconditioning for the classical proximal gradient method. The proposed scheme can be applied to additive…
This paper studies the asymptotic behavior of the constant step Stochastic Gradient Descent for the minimization of an unknown function F , defined as the expectation of a non convex, non smooth, locally Lipschitz random function. As the…
We consider in this work a system of two stochastic differential equations named the perturbed compositional gradient flow. By introducing a separation of fast and slow scales of the two equations, we show that the limit of the slow motion…
We prove a central limit theorem for a certain class of functions on sparse rank-one inhomogeneous random graphs endowed with additional i.i.d. edge and vertex weights. Our proof of the central limit theorem uses a perturbative form of…
In this work, we study the asymptotic randomness of an algorithmic estimator of the saddle point of a globally convex-concave and locally strongly-convex strongly-concave objective. Specifically, we show that the averaged iterates of a…
Ordinary differential equations obtained as limits of Markov processes appear in many settings. They may arise by scaling large systems, or by averaging rapidly fluctuating systems, or in systems involving multiple time-scales, by a…
Online minimization of an unknown convex function over the interval $[0,1]$ is considered under first-order stochastic bandit feedback, which returns a random realization of the gradient of the function at each query point. Without knowing…
The objective of this study is to investigate the limiting behavior of a subgraph counting process. The subgraph counting process we consider counts the number of subgraphs having a specific shape that exist outside an expanding ball as the…
This work addresses distributed optimization, where a network of agents wants to minimize a global strongly convex objective function. The global function can be written as a sum of local convex functions, each of which is associated with…
The fluctuation-dissipation theorem is a central result in statistical mechanics and is usually formulated for systems described by diffusion processes. In this paper, we propose a generalization for a wider class of stochastic processes,…
In this paper, we analyze the mirror descent algorithm for non-smooth optimization problems in which the objective function is relatively strongly convex, without relying on the standard Lipschitz continuity assumption commonly used in the…
We develop a new proximal-gradient method for minimizing the sum of a differentiable, possibly nonconvex, function plus a convex, possibly non differentiable, function. The key features of the proposed method are the definition of a…
Gradient descent is one of the most widely used iterative algorithms in modern statistical learning. However, its precise algorithmic dynamics in high-dimensional settings remain only partially understood, which has limited its broader…
In this report, we study decentralized stochastic optimization to minimize a sum of smooth and strongly convex cost functions when the functions are distributed over a directed network of nodes. In contrast to the existing work, we use…
Two time scale stochastic approximation algorithms emulate singularly perturbed deterministic differential equations in a certain limiting sense, i.e., the interpolated iterates on each time scale approach certain differential equations in…