Related papers: A decreasing step method for strongly oscillating …
Many relevant problems in the area of systems and control, such as controller synthesis, observer design and model reduction, can be viewed as optimization problems involving dynamical systems: for instance, maximizing performance in the…
The stochastic limit approximation method for ``rapid'' decay is presented, where the damping rate \gamma is comparable to the system frequency \Omega, i.e., \gamma \sim \Omega, whereas the usual stochastic limit approximation is applied…
It is known in \cite{beccari} that the standard explicit Euler-type scheme (such as the exponential Euler and the linear-implicit Euler schemes) with a uniform timestep, though computationally efficient, may diverge for the stochastic…
In this paper, we propose an implicit gradient descent algorithm for the classic $k$-means problem. The implicit gradient step or backward Euler is solved via stochastic fixed-point iteration, in which we randomly sample a mini-batch…
We first derive the exponential ergodicity of the stochastic theta method (STM) with $\theta \in (1/2,1]$ for monotone jump-diffusion stochastic ordinary differential equations (SODEs) under a dissipative condition. Then we establish the…
We develop an Euler-type method to predict the evolution of a time-dependent probability measure without explicitly learning an operator that governs its evolution. We use linearized optimal transport theory to prove that the measure-valued…
We propose an {\em implementable} numerical scheme for the discretization of linear-quadratic optimal control problems involving SDEs in higher dimensions with {\em control constraint}. For time discretization, we employ the implicit Euler…
The efficient simulation of the mean value of a non-linear functional of the solution to a linear stochastic partial differential equation (SPDE) with additive Gaussian noise is considered. A Galerkin finite element method is employed along…
This paper proposes a thorough theoretical analysis of Stochastic Gradient Descent (SGD) with non-increasing step sizes. First, we show that the recursion defining SGD can be provably approximated by solutions of a time inhomogeneous…
We consider a class of stochastic smooth convex optimization problems under rather general assumptions on the noise in the stochastic gradient observation. As opposed to the classical problem setting in which the variance of noise is…
We investigate the Randomized Stochastic Accelerated Gradient (RSAG) method, utilizing either constant or adaptive step sizes, for stochastic optimization problems with generalized smooth objective functions. Under relaxed affine variance…
We present stochastic variants of the exponential time differencing schemes for stiff stochastic differential equations. We derive three explicit schemes that offer better stability compared to Euler-Maruyama and Milstein's method, and…
We propose a modification of the standard linear implicit Euler integrator for the weak approximation of parabolic semilinear stochastic PDEs driven by additive space-time white noise. The new method can easily be combined with a finite…
This paper is concerned with the adaptive numerical treatment of stochastic partial differential equations. Our method of choice is Rothe's method. We use the implicit Euler scheme for the time discretization. Consequently, in each step, an…
This paper investigates the stochastic Cahn-Hilliard equation (SCHE) driven by additive space-time white noise. We first refine the analytical ergodic theory by proving that the continuum equation admits a unique invariant measure in the…
Stochastic Gradient Descent (SGD) is one of the simplest and most popular stochastic optimization methods. While it has already been theoretically studied for decades, the classical analysis usually required non-trivial smoothness…
In this work, we propose a new deep learning-based scheme for solving high dimensional nonlinear backward stochastic differential equations (BSDEs). The idea is to reformulate the problem as a global optimization, where the local loss…
We consider in this work the convergence of a split-step Euler type scheme (SSM) for the numerical simulation of interacting particle Stochastic Differential Equation (SDE) systems and McKean-Vlasov Stochastic Differential Equations…
The stochastic gradient descent (SGD) method is a widely used approach for solving stochastic optimization problems, but its convergence is typically slow. Existing variance reduction techniques, such as SAGA, improve convergence by…
Stochastic Gradient Descent (SGD) is one of the most popular algorithms in statistical and machine learning due to its computational and memory efficiency. Various averaging schemes have been proposed to accelerate the convergence of SGD in…