Related papers: Modified Equations for Stochastic Optimization
This paper proposes a thorough theoretical analysis of Stochastic Gradient Descent (SGD) with non-increasing step sizes. First, we show that the recursion defining SGD can be provably approximated by solutions of a time inhomogeneous…
This paper introduces time-continuous numerical schemes to simulate stochastic differential equations (SDEs) arising in mathematical finance, population dynamics, chemical kinetics, epidemiology, biophysics, and polymeric fluids. These…
This paper considers mean square error (MSE) analysis for stochastic gradient sampling algorithms applied to underdamped Langevin dynamics under a global convexity assumption. A novel discrete Poisson equation framework is developed to…
Variance reduction techniques are of crucial importance for the efficiency of Monte Carlo simulations in finance applications. We propose the use of neural SDEs, with control variates parameterized by neural networks, in order to learn…
There has been a great deal of recent interest in learning and approximation of functions that can be expressed as expectations of a given nonlinearity with respect to its random internal parameters. Examples of such representations include…
Many stochastic differential equations (SDEs) in the literature have a superlinearly growing nonlinearity in their drift or diffusion coefficient. Unfortunately, moments of the computationally efficient Euler-Maruyama approximation method…
We propose new numerical schemes for decoupled forward-backward stochastic differential equations (FBSDEs) with jumps, where the stochastic dynamics are driven by a $d$-dimensional Brownian motion and an independent compensated Poisson…
We consider the problem of obtaining effective representations for the solutions of linear, vector-valued stochastic differential equations (SDEs) driven by non-Gaussian pure-jump L\'evy processes, and we show how such representations lead…
Stochastic version of alternating direction method of multiplier (ADMM) and its variants (linearized ADMM, gradient-based ADMM) plays a key role for modern large scale machine learning problems. One example is the regularized empirical risk…
We propose a new numerical method for one dimensional stochastic differential equations (SDEs). The main idea of this method is based on a representation of a weak solution of a SDE with a time changed Brownian motion, dated back to Doeblin…
Stochastic Gradient Descent with a constant learning rate (constant SGD) simulates a Markov chain with a stationary distribution. With this perspective, we derive several new results. (1) We show that constant SGD can be used as an…
In the present article we study strong approximation of solutions of scalar stochastic differential equations (SDEs) with bounded and $\alpha$-H\"older continuous drift coefficient and constant diffusion coefficient at time point $1$.…
We consider the stochastic gradient descent (SGD) algorithm driven by a general stochastic sequence, including i.i.d noise and random walk on an arbitrary graph, among others; and analyze it in the asymptotic sense. Specifically, we employ…
This paper is devoted to a system of stochastic partial differential equations (SPDEs) that have a slow component driven by fractional Brownian motion (fBm) with the Hurst parameter $H >1/2$ and a fast component driven by fast-varying…
We study asymptotic error distributions associated with standard approximation scheme for one-dimensional stochastic differential equations driven by fractional Brownian motions. This problem was studied by, for instance, Gradinaru-Nourdin…
This article develops a stochastic differential equation (SDE) for modeling the temporal evolution of queue length dynamics at signalized intersections. Inspired by the observed quasiperiodic and self-similar characteristics of the queue…
In this paper, the stability behaviors of stochastic differential equations (SDEs) driven by time-changed Brownian motions are discussed. Based on the generalized Lyapunov method and stochastic analysis, necessary conditions are provided…
We develop a new continuous-time stochastic gradient descent method for optimizing over the stationary distribution of stochastic differential equation (SDE) models. The algorithm continuously updates the SDE model's parameters using an…
We consider stochastic semi-linear evolution equations which are driven by additive, spatially correlated, Wiener noise, and in particular consider problems of heat equation (analytic semigroup) and damped-driven wave equations (bounded…
The approximation of invariant measures for nonlinear ergodic stochastic differential equations (SDEs) is a central problem in scientific computing, with important applications in stochastic sampling, physics, and ecology. We first propose…