Related papers: Error distributions for random grid approximations…
In this paper, we study the convergence properties of the Stochastic Gradient Descent (SGD) method for finding a stationary point of a given objective function $J(\cdot)$. The objective function is not required to be convex. Rather, our…
Developing efficient numerical algorithms for the solution of high dimensional random Partial Differential Equations (PDEs) has been a challenging task due to the well-known curse of dimensionality. We present a new solution framework for…
We consider stochastic approximations of sampling algorithms, such as Stochastic Gradient Langevin Dynamics (SGLD) and the Random Batch Method (RBM) for Interacting Particle Dynamcs (IPD). We observe that the noise introduced by the…
This paper concerns error bounds for recursive equations subject to Markovian disturbances. Motivating examples abound within the fields of Markov chain Monte Carlo (MCMC) and Reinforcement Learning (RL), and many of these algorithms can be…
In this paper, we investigate the theoretical properties of stochastic gradient descent (SGD) for statistical inference in the context of nonconvex optimization problems, which have been relatively unexplored compared to convex settings.…
Stochastic Gradient Descent (SGD) is an important algorithm in machine learning. With constant learning rates, it is a stochastic process that, after an initial phase of convergence, generates samples from a stationary distribution. We show…
We provide a abstract framework to prove total variation convergence result with arbitrary rate for numerical scheme for SDE. In particular we show that under standard weak approximation properties of scheme such as Euler we can obtain…
Motivated by the simulation of stable random fields, we consider the issue of discrete approximations of independently scattered stable noise. Two approaches are proposed: grid approximations available when the underlying space is $\bbR^d$…
This paper develops a framework for the error analysis in nonparametric model fitting of fractional stochastic differential equations based on discrete observations. We identify and quantify the main error sources -- time discretization,…
Stochastic gradient descent (SGD) and its variants are widely used and highly effective optimization methods in machine learning, especially for neural network training. By using a single datum or a small subset of the data, selected…
For a class of symmetric random matrices whose entries are martingale differences adapted to an increasing filtration, we prove that under a Lindeberg-like condition, the empirical spectral distribution behaves asymptotically similarly to a…
This paper presents an algorithmic framework for solving unconstrained stochastic optimization problems using only stochastic function evaluations. We employ central finite-difference based gradient estimation methods to approximate the…
We present an abstract concept for the error analysis of numerical schemes for semilinear stochastic partial differential equations (SPDEs) and demonstrate its usefulness by proving the strong convergence of a Milstein-Galerkin finite…
Stochastic rounding (SR) offers an alternative to the deterministic IEEE-754 floating-point rounding modes. In some applications such as PDEs, ODEs and neural networks, SR empirically improves the numerical behavior and convergence to…
We provide the first stochastic convergence rates for a family of adaptive quadrature rules used to normalize the posterior distribution in Bayesian models. Our results apply to the uniform relative error in the approximate posterior…
This paper considers the distributed sparse identification problem over wireless sensor networks such that all sensors cooperatively estimate the unknown sparse parameter vector of stochastic dynamic systems by using the local information…
We develop a novel stochastic primal dual splitting method with Bregman distances for solving a structured composite problems involving infimal convolutions in non-Euclidean spaces. The sublinear convergence in expectation of the…
In this paper we provide necessary and sufficient conditions for the mean square approximation of a random field with an ortho-martingale. The conditions are formulated in terms of projective criteria. Applications are given to linear and…
In this article, we consider convergence of stochastic gradient descent schemes (SGD), including momentum stochastic gradient descent (MSGD), under weak assumptions on the underlying landscape. More explicitly, we show that on the event…
We establish convergence theorems for Riemannian stochastic gradient descents in which the underlying probability spaces vary from iteration to iteration. As applications, we deduce convergence results for Riemannian stochastic gradient…