Related papers: Sharp convex generalizations of stochastic Gronwal…
Two years ago, Conlon and Gowers, and Schacht proved general theorems that allow one to transfer a large class of extremal combinatorial results from the deterministic to the probabilistic setting. Even though the two papers solve the same…
This paper proposes a novel Gronwall inequality-based method for transient stability assessment for power systems. The challenges of applying such methods to power systems are how to construct the differential inequality and how to treat…
It is well known that there is a strong connection between entropy inequalities and submodularity, since the entropy of a collection of random variables is a submodular function. Unifying frameworks for information inequalities arising from…
In this thesis, we extend the recently introduced theory of stochastic modified equations (SMEs) for stochastic gradient optimization algorithms. In Ch. 3 we study time-inhomogeneous SDEs driven by Brownian motion. For certain SDEs we prove…
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss functions, it is violated for cases where the objective…
The existence of random dynamical systems for McKean--Vlasov SDEs is established. This is approached by considering the joint dynamics of the corresponding nonlinear Fokker-Planck equation governing the law of the system and the underlying…
This paper studies finite element approximations of the stochastic Allen-Cahn equation with gradient-type multiplicative noises that are white in time and correlated in space. The sharp interface limit as the parameter $\epsilon \rightarrow…
This study addresses the inverse problem of parameter estimation for Stochastic Differential Equations (SDEs) by minimizing a regularized discrepancy functional via Stochastic Gradient Descent (SGD). To achieve computational efficiency, we…
Averaging iterations of Stochastic Gradient Descent (SGD) have achieved empirical success in training deep learning models, such as Stochastic Weight Averaging (SWA), Exponential Moving Average (EMA), and LAtest Weight Averaging (LAWA).…
Many modern large-scale machine learning problems benefit from decentralized and stochastic optimization. Recent works have shown that utilizing both decentralized computing and local stochastic gradient estimates can outperform…
We study hypoelliptic stochastic differential equations (SDEs) and their connection to degenerate-elliptic boundary value problems on bounded or unbounded domains. In particular, we provide probabilistic conditions that guarantee that the…
The stochastic Frank-Wolfe method has recently attracted much general interest in the context of optimization for statistical and machine learning due to its ability to work with a more general feasible region. However, there has been a…
We give a new take on the error analysis of approximations of stochastic differential equations (SDEs), utilizing and developing the stochastic sewing lemma of L\^e (2020). This approach allows one to exploit regularization by noise effects…
The Euler scheme is one of the standard schemes to obtain numerical approximations of stochastic differential equations (SDEs). Its convergence properties are well-known in the case of globally Lipschitz continuous coefficients. However, in…
We analyze stochastic algorithms for optimizing nonconvex, nonsmooth finite-sum problems, where the nonconvex part is smooth and the nonsmooth part is convex. Surprisingly, unlike the smooth case, our knowledge of this fundamental problem…
Stochastic differential equations have proved to be a valuable governing framework for many real-world systems which exhibit ``noise'' or randomness in their evolution. One quality of interest in such systems is the shape of their…
We present a unified theorem for the convergence analysis of stochastic gradient algorithms for minimizing a smooth and convex loss plus a convex regularizer. We do this by extending the unified analysis of Gorbunov, Hanzely \& Richt\'arik…
Recent studies have provided both empirical and theoretical evidence illustrating that heavy tails can emerge in stochastic gradient descent (SGD) in various scenarios. Such heavy tails potentially result in iterates with diverging…
I propose a novel framework that integrates stochastic differential equations (SDEs) with deep generative models to improve uncertainty quantification in machine learning applications involving structured and temporal data. This approach,…
We propose graph-dependent implicit regularisation strategies for distributed stochastic subgradient descent (Distributed SGD) for convex problems in multi-agent learning. Under the standard assumptions of convexity, Lipschitz continuity,…