Related papers: Sharp convex generalizations of stochastic Gronwal…
In the recent article [Hairer, M., Hutzenthaler, M., Jentzen, A., Loss of regularity for Kolmogorov equations, Ann. Probab. 43 (2015), no. 2, 468--527] it has been shown that there exist stochastic differential equations (SDEs) with…
Recent years have witnessed significant progress in developing effective training and fast sampling techniques for diffusion models. A remarkable advancement is the use of stochastic differential equations (SDEs) and their…
In this paper, existence and uniqueness are proved for path-dependent McKean-Vlasov type SDEs with integrability conditions. Gradient estimates and Harnack type inequalities are derived in the case that the coefficients are Dini continuous…
Uniform stability is a notion of algorithmic stability that bounds the worst case change in the model output by the algorithm when a single data point in the dataset is replaced. An influential work of Hardt et al. (2016) provides strong…
The aim of this paper is to obtain convergence in mean in the uniform topology of piecewise linear approximations of Stochastic Differential Equations (SDEs) with $C^1$ drift and $C^2$ diffusion coefficients with uniformly bounded…
In this article we study (possibly degenerate) stochastic differential equations (SDE) with irregular (or discontiuous) coefficients, and prove that under certain conditions on the coefficients, there exists a unique almost everywhere…
We introduce a general framework for nonlinear stochastic gradient descent (SGD) for the scenarios when gradient noise exhibits heavy tails. The proposed framework subsumes several popular nonlinearity choices, like clipped, normalized,…
We consider the problem of convergence to a saddle point of a concave-convex function via gradient dynamics. Since first introduced by Arrow, Hurwicz and Uzawa in [1] such dynamics have been extensively used in diverse areas, there are,…
Many stochastic differential equations (SDEs) in the literature have a superlinearly growing nonlinearity in their drift or diffusion coefficient. Unfortunately, moments of the computationally efficient Euler-Maruyama approximation method…
We prove the following version generalization of the Gronwall inequality: Let $\mathbf X$ be a Banach space and $U\subset \mathbf X$ an open convex set in $\mathbf X$. Let $f,g\colon [a,b]\times U\to \mathbf X$ be continuous functions and…
Stochastic differential equations (SDEs) have been shown recently to characterize well the dynamics of training machine learning models with SGD. When the generalization error of the SDE approximation closely aligns with that of SGD in…
Our work is part of the close link between continuous-time dissipative dynamical systems and optimization algorithms, and more precisely here, in the stochastic setting. We aim to study stochastic convex minimization problems through the…
Diffusion models, typically formulated as discretizations of stochastic differential equations (SDEs), have achieved state-of-the-art performance in generative tasks. However, their theoretical analysis often involves complex proofs. In…
We analyze the global and local behavior of gradient-like flows under stochastic errors towards the aim of solving convex optimization problems with noisy gradient input. We first study the unconstrained differentiable convex case, using a…
Since the pioneering work of Gerhard Gruss dating back to 1935, Gruss's inequality and, more generally, Gruss-type bounds for covariances have fascinated researchers and found numerous applications in areas such as economics, insurance,…
A number of geometric inequalities for convex sets arising from Brunn's concavity principle have recently been shown to yield local stochastic formulations. Comparatively, there has been much less progress towards stochastic forms of…
Stochastic gradient descent with momentum (SGDM) methods have become fundamental optimization tools in machine learning, combining the computational efficiency of stochastic gradients with the acceleration benefits of momentum. Despite…
Several recent works have explored stochastic gradient methods for variational inference that exploit the geometry of the variational-parameter space. However, the theoretical properties of these methods are not well-understood and these…
This paper presents a new type of Gronwall-Bellman inequality, which arises from a class of integral equations with a mixture of nonsingular and singular integrals. The new idea is to use a binomial function to combine the known…
While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we…