Related papers: Quantitative Fluctuation Analysis for Continuous-T…
Consider generalized adapted stochastic integrals with respect to independently scattered random measures with second moments. We use a decoupling technique, known as the "principle of conditioning", to study their stable convergence…
This paper presents the convergence analysis of the spatial finite difference method (FDM) for the stochastic Cahn--Hilliard equation with Lipschitz nonlinearity and multiplicative noise. Based on fine estimates of the discrete Green…
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss functions, it is violated for cases where the objective…
This article considers the stochastic optimal control of discrete-time linear systems subject to (possibly) unbounded stochastic disturbances, hard constraints on the manipulated variables, and joint chance constraints on the states. A…
Stochastic Gradient Descent (SGD) is a fundamental algorithm in machine learning, representing the optimization backbone for training several classic models, from regression to neural networks. Given the recent practical focus on…
Gradient optimization algorithms using epochs, that is those based on stochastic gradient descent without replacement (SGDo), are predominantly used to train machine learning models in practice. However, the mathematical theory of SGDo and…
Two time scale stochastic approximation algorithms emulate singularly perturbed deterministic differential equations in a certain limiting sense, i.e., the interpolated iterates on each time scale approach certain differential equations in…
Learning in deep models using Bayesian methods has generated significant attention recently. This is largely because of the feasibility of modern Bayesian methods to yield scalable learning and inference, while maintaining a measure of…
The diffusion approximation of stochastic gradient descent (SGD) in current literature is only valid on a finite time interval. In this paper, we establish the uniform-in-time diffusion approximation of SGD, by only assuming that the…
We analyze the dynamics of streaming stochastic gradient descent (SGD) in the high-dimensional limit when applied to generalized linear models and multi-index models (e.g. logistic regression, phase retrieval) with general data-covariance.…
We study the scaling limits of stochastic gradient descent (SGD) with constant step-size in the high-dimensional regime. We prove limit theorems for the trajectories of summary statistics (i.e., finite-dimensional functions) of SGD as the…
This paper studies the high-dimensional scaling limits of online stochastic gradient descent (SGD). Building on the recent work of Ben Arous, Gheissari, and Jagannath on the effective dynamics of SGD, we study the critical scaling regime of…
Stochastic gradient descent (SGD) has been a go-to algorithm for nonconvex stochastic optimization problems arising in machine learning. Its theory however often requires a strong framework to guarantee convergence properties. We hereby…
In 2010, Shiffman and Zelditch proved a central limit theorem (CLT) for smooth statistics of Gaussian random zeros in codimension one over compact K\"ahler manifolds. They raised the question of whether this result admits a two-fold…
In this paper, we study a stochastic strongly convex optimization problem and propose three classes of variable sample-size stochastic first-order methods including the standard stochastic gradient descent method, its accelerated variant,…
The purpose of this paper is to analyze the distribution distance between random vectors derived from the magnitude of the analytic wavelet transform of the squared envelopes of Gaussian processes and their large-scale limits. When the…
This study deals with continuous limits of interacting one-dimensional diffusive systems, arising from stochastic distortions of discrete curves with various kinds of coding representations. These systems are essentially of a…
We develop a general technique for proving convergence of repeated quantum interactions to the solution of a quantum stochastic differential equation. The wide applicability of the method is illustrated in a variety of examples. Our main…
In this paper, we study the convergence properties of the Stochastic Gradient Descent (SGD) method for finding a stationary point of a given objective function $J(\cdot)$. The objective function is not required to be convex. Rather, our…
In this paper, we introduce the $\sigma$-antithetic multilevel Monte Carlo (MLMC) estimator for a multi-dimensional diffusion which is an extended version of the original antithetic MLMC one introduced by Giles and Szpruch \cite{a}. Our aim…