Related papers: On the stability of the stochastic gradient Langev…
Adaptive Langevin dynamics is a method for sampling the Boltzmann-Gibbs distribution at prescribed temperature in cases where the potential gradient is subject to stochastic perturbation of unknown magnitude. The method replaces the…
A model has two main aims: predicting the behavior of a physical system and understanding its nature, that is how it works, at some desired level of abstraction. A promising recent approach to model building consists in deriving a…
We introduce a novel and efficient algorithm called the stochastic approximate gradient descent (SAGD), as an alternative to the stochastic gradient descent for cases where unbiased stochastic gradients cannot be trivially obtained.…
How can we learn the laws underlying the dynamics of stochastic systems when their trajectories are sampled sparsely in time? Existing methods either require temporally resolved high-frequency observations, or rely on geometric arguments…
We introduce a scheme for deriving an optimally-parametrised Langevin dynamics of few collective variables from data generated in molecular dynamics simulations. The drift and the position-dependent diffusion profiles governing the Langevin…
We establish a data-dependent notion of algorithmic stability for Stochastic Gradient Descent (SGD), and employ it to develop novel generalization bounds. This is in contrast to previous distribution-free algorithmic stability results for…
It is well known that adding any skew symmetric matrix to the gradient of Langevin dynamics algorithm results in a non-reversible diffusion with improved convergence rate. This paper presents a gradient algorithm to adaptively optimize the…
We study the Stochastic Gradient Langevin Dynamics (SGLD) algorithm for non-convex optimization. The algorithm performs stochastic gradient descent, where in each step it injects appropriately scaled Gaussian noise to the update. We analyze…
Motivated by an application to empirical Bayes learning in high-dimensional regression, we study a class of Langevin diffusions in a system with random disorder, where the drift coefficient is driven by a parameter that continuously adapts…
Using the method of Krylov's estimates, we prove the existence of weak solutions of stochastic differential equations driven by purely discontinuous Levy processes satisfying an additional assumption. The diffusion coefficient is assumed to…
In this note, we consider the construction of a one-dimensional stable Langevin type process confined in the upper half-plane and submitted to reflective-diffusive boundary conditions whenever the particle position hits 0. We show that two…
Metastable transitions in Langevin dynamics can exhibit rich behaviors that are markedly different from its overdamped limit. In addition to local alterations of the transition path geometry, more fundamental global changes may exist. For…
The viability of a variant of numerical stochastic perturbation theory, where the Langevin equation is replaced by the SMD algorithm, is examined. In particular, the convergence of the process to a unique stationary state is rigorously…
Our work is motivated by a desire to study the theoretical underpinning for the convergence of stochastic gradient type algorithms widely used for non-convex learning tasks such as training of neural networks. The key insight, already…
Stochastic gradient Langevin dynamics (SGLD) has gained the attention of optimization researchers due to its global optimization properties. This paper proves an improved convergence property to local minimizers of nonconvex objective…
Langevin dynamics has found a large number of applications in sampling, optimization and estimation. Preconditioning the gradient in the dynamics with the covariance - an idea that originated in literature related to solving estimation and…
We propose a method for efficiently incorporating constraints into a stochastic gradient Langevin framework for the training of deep neural networks. Constraints allow direct control of the parameter space of the model. Appropriately…
Continuous-time models provide important insights into the training dynamics of optimization algorithms in deep learning. In this work, we establish a non-asymptotic convergence analysis of stochastic gradient Langevin dynamics (SGLD),…
We investigate the asymptotic distributions of periodically driven anharmonic Langevin systems. Utilizing the underlying $SL_2$ symmetry of the Langevin dynamics, we develop a perturbative scheme in which the effect of periodic driving can…
Obtaining coarse-grained models that accurately incorporate finite-size effects is an important open challenge in the study of complex, multi-scale systems. We apply Langevin regression, a recently developed method for finding stochastic…