Related papers: A weak convergence approach to large deviations fo…

We analyze the finite sample regret of a decreasing step size stochastic gradient algorithm. We assume correlated noise and use a perturbed Lyapunov function as a systematic approach for the analysis. Finally we analyze the escape time of…

Stochastic approximation algorithms are iterative procedures which are used to approximate a target value in an environment where the target is unknown and direct observations are corrupted by noise. These algorithms are useful, for…

We study the so-called two-time-scale stochastic approximation, a simulation-based approach for finding the roots of two coupled nonlinear operators. Our focus is to characterize its finite-time performance in a Markov setting, which often…

Time-irreversible stochastic processes are frequently used in natural sciences to explain non-equilibrium phenomena and to design efficient stochastic algorithms. Our main goal in this thesis is to analyse their dynamics by means of large…

Large deviation theory is a branch of probability theory that is devoted to a study of the "rate" at which empirical estimates of various quantities converge to their true values. The object of study in this paper is the rate at which…

Scalable algorithms of posterior approximation allow Bayesian nonparametrics such as Dirichlet process mixture to scale up to larger dataset at fractional cost. Recent algorithms, notably the stochastic variational inference performs local…

We prove pathwise large deviation principles of slow variables in slow-fast systems in the limit of time-scale separation tending to infinity. In the limit regime we consider, the convergence of the slow variable to its deterministic limit…

Stochastic Gradient Descent with a constant learning rate (constant SGD) simulates a Markov chain with a stationary distribution. With this perspective, we derive several new results. (1) We show that constant SGD can be used as an…

In this paper we develop the large deviations principle and a rigorous mathematical framework for asymptotically efficient importance sampling schemes for general, fully dependent systems of stochastic differential equations of slow and…

The convergence of stochastic gradient descent is highly dependent on the step-size, especially on non-convex problems such as neural network training. Step decay step-size schedules (constant and then cut) are widely used in practice…

In empirical risk optimization, it has been observed that stochastic gradient implementations that rely on random reshuffling of the data achieve better performance than implementations that rely on sampling the data uniformly. Recent works…

The asymptotic analysis of a class of stochastic partial differential equations (SPDEs) with fully locally monotone coefficients covering a large variety of physical systems, a wide class of quasilinear SPDEs and a good number of fluid…

In this paper, we establish the large deviation principle for 3D stochastic primitive equations with small perturbation multiplicative noise. The proof is mainly based on the weak convergence approach.

In this paper we prove large and moderate deviations principles for the recursive kernel estimators of a probability density function defined by the stochastic approximation algorithm introduced by Mokkadem et al. [2009. The stochastic…

Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a…

This paper provides convergence analysis for the approximation of a class of path-dependent functionals underlying a continuous stochastic process. In the first part, given a sequence of weak convergent processes, we provide a sufficient…

Stochastic approximation is a foundation for many algorithms found in machine learning and optimization. It is in general slow to converge: the mean square error vanishes as $O(n^{-1})$. A deterministic counterpart known as quasi-stochastic…

This work is concerned with the large deviation principle for a family of slow-fast systems perturbed by infinite-dimensional mixed fractional Brownian motion with Hurst parameter $H\in(\frac12,1)$. We adopt the weak convergence method…

Stochastic gradient algorithms have been the main focus of large-scale learning problems and they led to important successes in machine learning. The convergence of SGD depends on the careful choice of learning rate and the amount of the…

In this paper, we introduce a new approach to proving the convergence of the Stochastic Approximation (SA) and the Stochastic Gradient Descent (SGD) algorithms. The new approach is based on a concept called GSLLN (Generalized Strong Law of…