Related papers: Modified Equations for Stochastic Optimization

Overparameterized stochastic differential equation (SDE) models have achieved remarkable success in various complex environments, such as PDE-constrained optimization, stochastic control and reinforcement learning, financial engineering,…

The Dyson Brownian Motion (DBM) describes the stochastic evolution of $N$ points on the line driven by an applied potential, a Coulombic repulsion and identical, independent Brownian forcing at each point. We use an explicit tamed Euler…

In this paper numerical methods for solving stochastic differential equations with Markovian switching (SDEwMSs) are developed by pathwise approximation. The proposed family of strong predictor-corrector Euler-Maruyama methods is designed…

Optimization algorithms with momentum, e.g., (ADAM), have been widely used for building deep learning models due to the faster convergence rates compared with stochastic gradient descent (SGD). Momentum helps accelerate SGD in the relevant…

We extend the unified kernel framework for transport equations and Koopman eigenfunctions, developed in previous work by the authors for deterministic systems, to stochastic differential equations (SDEs). In the deterministic setting, three…

Stemmed from the derivation of the optimal control to a stochastic linear-quadratic control problem with Markov jumps, we study one kind of backward stochastic differential equations (BSDEs) that the generator f is affected by a Markovian…

Many global optimization algorithms of the memetic variety rely on some form of stochastic search, and yet they often lack a sound probabilistic basis. Without a recourse to the powerful tools of stochastic calculus, treading the fine…

We consider a class of backward stochastic differential equations (BSDEs) driven by Brownian motion and Poisson random measure, and subject to constraints on the jump component. We prove the existence and uniqueness of the minimal solution…

Stochastic differential equations (SDEs), which models uncertain phenomena as the time evolution of random variables, are exploited in various fields of natural and social sciences such as finance. Since SDEs rarely admit analytical…

In this work we investigate a 1D evolution equation involving a divergence form operator where the diffusion coefficient inside the divergence is changing sign, as in models for metamaterials.We focus on the construction of a fundamental…

In this paper, we study dimension reduction techniques for large-scale controlled stochastic differential equations (SDEs). The drift of the considered SDEs contains a polynomial term satisfying a one-sided growth condition. Such…

Consider the following stochastic differential equation for $(X_t)_{t\ge 0}$ on $\mathbb R^d$ and its Euler-Maruyama (EM) approximation $(Y_{t_n})_{n\in \mathbb Z^+}$: \begin{align*} &d X_t=b( X_t) d t+\sigma(X_t) d B_t, \\ &…

This paper investigates a non-autonomous slow-fast system, which is generalized by stochastic differential equations (SDEs) with locally Lipschitz coefficients, subjected to standard Brownian motion (Bm) and fractional Brownian motion (fBm)…

Neural stochastic differential equation model with a Brownian motion term can capture epistemic uncertainty of deep neural network from the perspective of a dynamical system. The goal of this paper is to improve the convergence rate of the…

We propose new continuous-time formulations for first-order stochastic optimization algorithms such as mini-batch gradient descent and variance-reduced methods. We exploit these continuous-time models, together with simple Lyapunov analysis…

Recent advancements in deep learning optimization have introduced new algorithms, such as Schedule-Free optimizers, AdEMAMix, MARS and Lion which modify traditional momentum mechanisms. In a separate line of work, theoretical acceleration…

In this paper, we investigate large-scale linear systems driven by a fractional Brownian motion (fBm) with Hurst parameter $H\in [1/2, 1)$. We interpret these equations either in the sense of Young ($H>1/2$) or Stratonovich ($H=1/2$).…

We consider stochastic differential equations (SDEs) driven by a fractional Brownian motion with a drift coefficient that is allowed to be arbitrarily close to criticality in a scaling sense. We develop a comprehensive solution theory that…

We develope a perturbation theory for stochastic differential equations (SDEs) by which we mean both stochastic ordinary differential equations (SODEs) and stochastic partial differential equations (SPDEs). In particular, we estimate the $…

We present a novel control variate technique for enhancing the efficiency of Monte Carlo (MC) estimation of expectations involving solutions to stochastic differential equations (SDEs). Our method integrates a primary fine-time-step…