Related papers: A weak convergence approach to large deviations fo…
This work develops new results for stochastic approximation algorithms. The emphases are on treating algorithms and limits with discontinuities. The main ingredients include the use of differential inclusions, set-valued analysis, and…
Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential…
In this paper, we study the asymptotic behavior of randomly perturbed path-dependent stochastic differential equations with small parameter $\vartheta_{\varepsilon}$, when $\varepsilon \rightarrow 0$, $\vartheta_\varepsilon$ goes to $0$.…
Averaging is an important method to extract effective macroscopic dynamics from complex systems with slow modes and fast modes. This article derives an averaged equation for a class of stochastic partial differential equations without any…
We prove a large-deviation principle (LDP) for the sample paths of jump Markov processes in the small noise limit when, possibly, all the jump rates vanish uniformly, but slowly enough, in a region of the state space. We further discuss the…
We establish a large deviation principle for the solutions of a class of stochastic partial differential equations with non-Lipschitz continuous coefficients. As an application, the large deviation principle is derived for super-Brownian…
This work concerns generalized backward stochastic differential equations, which are coupled with a family of reflecting diffusion processes. First of all, we establish the large deviation principle for forward stochastic differential…
We study stochastic optimization of nonconvex loss functions, which are typical objectives for training neural networks. We propose stochastic approximation algorithms which optimize a series of regularized, nonlinearized losses on large…
Stochastic gradient descent (SGD) is a popular algorithm for minimizing objective functions that arise in machine learning. For constant step-sized SGD, the iterates form a Markov chain on a general state space. Focusing on a class of…
An usual problem in statistics consists in estimating the minimizer of a convex function. When we have to deal with large samples taking values in high dimensional spaces, stochastic gradient algorithms and their averaged versions are…
We introduce and test an algorithm that adaptively estimates large deviation functions characterizing the fluctuations of additive functionals of Markov processes in the long-time limit. These functions play an important role for predicting…
In this paper, we consider McKean-Vlasov stochastic differential equations (MVSDEs) driven by L\'evy noise. By identifying the right equations satisfied by the solutions of the MVSDEs with shifted driving L\'evy noise, we build up a…
We study the large deviations of Markov chains under the sole assumption that the state space is discrete. In particular, we do not require any of the usual irreducibility and exponential tightness assumptions. Using subadditive arguments,…
To investigate the complex dynamics of a biological neuron that is subject to small random perturbations we can use stochastic neuron models. While many techniques have already been developed to study properties of such models, especially…
Stochastic gradient methods enable learning probabilistic models from large amounts of data. While large step-sizes (learning rates) have shown to be best for least-squares (e.g., Gaussian noise) once combined with parameter averaging,…
An algorithm of searching a zero of an unknown undimensional function is considered, measured at a point x with some error. The step sizes are random positive values and are calculated according to the rule: if two consecutive iterations…
Stochastic difference-of-convex (DC) optimization is prevalent in numerous machine learning applications, yet its convergence properties under small batch sizes remain poorly understood. Existing methods typically require large batches or…
In this paper we address the convergence of stochastic approximation when the functions to be minimized are not convex and nonsmooth. We show that the "mean-limit" approach to the convergence which leads, for smooth problems, to the ODE…
We consider a system of stochastic interacting particles in $\mathbb{R}^d$ and we describe large deviations asymptotics in a joint mean-field and small-noise limit. Precisely, a large deviations principle (LDP) is established for the…
The performance of standard stochastic approximation implementations can vary significantly based on the choice of the steplength sequence, and in general, little guidance is provided about good choices. Motivated by this gap, in the first…