Related papers: Modified Equations for Stochastic Optimization
We study the dynamics of a continuous-time model of the Stochastic Gradient Descent (SGD) for the least-square problem. Indeed, pursuing the work of Li et al. (2019), we analyze Stochastic Differential Equations (SDEs) that model SGD either…
In this paper, we study the numerical approximation of a general second order semilinear stochastic partial differential equation (SPDE) driven by a additive fractional Brownian motion (fBm) with Hurst parameter $H>\frac 12$ and Poisson…
Constructions of numerous approximate sampling algorithms are based on the well-known fact that certain Gibbs measures are stationary distributions of ergodic stochastic differential equations (SDEs) driven by the Brownian motion. However,…
In this article we propose a new explicit Euler-type approximation method for stochastic differential equations (SDEs). In this method, Brownian increments in the recursion of the Euler method are replaced by suitable bounded functions of…
We address the weak numerical solution of stochastic differential equations driven by independent Brownian motions (SDEs for short). This paper develops a new methodology to design adaptive strategies for determining automatically the…
We construct a nonstandard finite difference numerical scheme to approximate stochastic differential equations (SDEs) using the idea of weighed step introduced by R.E. Mickens. We prove the strong convergence of our scheme under locally…
We present a novel model Graph Neural Stochastic Differential Equations (Graph Neural SDEs). This technique enhances the Graph Neural Ordinary Differential Equations (Graph Neural ODEs) by embedding randomness into data representation using…
We study the stochastic optimization problem from a continuous-time perspective, with a focus on the Stochastic Gradient Descent with Momentum (SGDM) method. We show that the trajectory of SGDM, despite its \emph{stochastic} nature,…
We study strong approximation of scalar additive noise driven stochastic differential equations (SDEs) at time point $1$ in the case that the drift coefficient is bounded and has Sobolev regularity $s\in(0,1)$. Recently, it has been shown…
In this article, we consider the so-called modified Euler scheme for stochastic differential equations (SDEs) driven by fractional Brownian motions (fBm) with Hurst parameter $\frac13<H<\frac12$. This is a first-order time-discrete…
We study pathwise approximation of scalar stochastic differential equations at a single time point or globally in time by means of methods that are based on finitely many observations of the driving Brownian motion. We prove lower error…
Approximating Stochastic Gradient Descent (SGD) as a Stochastic Differential Equation (SDE) has allowed researchers to enjoy the benefits of studying a continuous optimization trajectory while carefully preserving the stochasticity of SGD.…
Non-uniform sampling arises when an experimenter does not have full control over the sampling characteristics of the process under investigation. Moreover, it is introduced intentionally in algorithms such as Bayesian optimization and…
A framework previously introduced in [3] for solving a sequence of stochastic optimization problems with bounded changes in the minimizers is extended and applied to machine learning problems such as regression and classification. The…
Distributed stochastic optimization algorithms can simultaneously process large-scale datasets, significantly accelerating model training. However, their effectiveness is often hindered by the sparsity of distributed networks and data…
We introduce a tamed exponential time integrator which exploits linear terms in both the drift and diffusion for Stochastic Differential Equations (SDEs) with a one sided globally Lipschitz drift term. Strong convergence of the proposed…
Continuous-time approximation of Stochastic Gradient Descent (SGD) is a crucial tool to study its escaping behaviors from stationary points. However, existing stochastic differential equation (SDE) models fail to fully capture these…
We consider a class of stochastic gradient optimization schemes. Assuming that the objective function is strongly convex, we prove weak error estimates which are uniform in time for the error between the solution of the numerical scheme,…
We present a method for approximating solutions of Stochastic Differential Equations (SDEs) with arbitrary rates. This approximation is derived for bounded and measurable test functions. Specifically, we demonstrate that, leveraging the…
This paper investigates the approximation of stochastic delay differential equations (SDDEs) via the backward Euler-Maruyama (BEM) method under generalized monotonicity and Khasminskii-type conditions in the infinite horizon. First, by…