Related papers: Modified Equations for Stochastic Optimization

Large-scale constrained optimization problems are at the core of many tasks in control, signal processing, and machine learning. Notably, problems with functional constraints arise when, beyond a performance{\nobreakdash-}centric goal…

We construct a reduced, data-driven, parameter dependent effective Stochastic Differential Equation (eSDE) for electric-field mediated colloidal crystallization using data obtained from Brownian Dynamics Simulations. We use Diffusion Maps…

Models defined by stochastic differential equations (SDEs) allow for the representation of random variability in dynamical systems. The relevance of this class of models is growing in many applied research areas and is already a standard…

Our work is part of the close link between continuous-time dissipative dynamical systems and optimization algorithms, and more precisely here, in the stochastic setting. We aim to study stochastic convex minimization problems through the…

This paper aims to investigate the numerical approximation of a general second order parabolic stochastic partial differential equation(SPDE) driven by multiplicative and additive noise. Our main interest is on such SPDEs where the…

In this paper we deal with pointwise approximation of solutions of stochastic differential equations (SDEs) driven by infinite dimensional Wiener process with additional jumps generated by Poisson random measure. The further investigations…

Stochastic gradient descent (SGD), which dates back to the 1950s, is one of the most popular and effective approaches for performing stochastic optimization. Research on SGD resurged recently in machine learning for optimizing convex loss…

We study the performance of stochastic gradient descent (SGD) on smooth and strongly-convex finite-sum optimization problems. In contrast to the majority of existing theoretical works, which assume that individual functions are sampled with…

We construct Wong--Zakai approximations of time--inhomogeneous stochastic differential equations with regime switching (RSSDEs), and provide a convergence rate. %Given a family of finite-variation processes…

The signature is a collection of iterated integrals describing the "shape" of a path. It appears naturally in the Taylor expansions of controlled differential equations and, as a consequence, is arguably the central object within rough path…

This paper establishes a discretization scheme for a large class of stochastic differential equations driven by a time-changed Brownian motion with drift, where the time change is given by a general inverse subordinator. The scheme involves…

The gradient noise (GN) in the stochastic gradient descent (SGD) algorithm is often considered to be Gaussian in the large data regime by assuming that the \emph{classical} central limit theorem (CLT) kicks in. This assumption is often made…

We present preconditioned stochastic gradient descent (SGD) algorithms for the $\ell_1$ minimization problem $\min_{x}\|A x - b\|_1$ in the overdetermined case, where there are far more constraints than variables. Specifically, we have $A…

We construct a new family of random permutons, called skew Brownian permuton, which describes the limits of several models of random constrained permutations. This family is parametrized by two real parameters. For a specific choice of the…

Stochastic symmetries and related invariance properties of finite dimensional SDEs driven by general c\`adl\`ag semimartingales taking values in Lie groups are defined and investigated. In order to enlarge the class of possible symmetries…

Stochastic gradient descent (SGD) is the workhorse of large-scale learning, yet classical analyses rely on assumptions that can be either too strong (bounded variance) or too coarse (uniform noise). The expected smoothness (ES) condition…

We are concerned with homogenization of stochastic differential equations (SDE) with stationary coefficients driven by Poisson random measures and Brownian motions in the critical case, that is when the limiting equation admits both a…

In the recent article [Jentzen, A., M\"uller-Gronbach, T., and Yaroslavtseva, L., Commun. Math. Sci., 14(6), 1477--1500, 2016] it has been established that for every arbitrarily slow convergence speed and every natural number $d \in…

Various gradient compression schemes have been proposed to mitigate the communication cost in distributed training of large scale machine learning models. Sign-based methods, such as signSGD, have recently been gaining popularity because of…

We consider stochastic approximations of sampling algorithms, such as Stochastic Gradient Langevin Dynamics (SGLD) and the Random Batch Method (RBM) for Interacting Particle Dynamcs (IPD). We observe that the noise introduced by the…