Related papers: On the implicit regularization of Langevin dynamic…
We study the stability of regularization by projection for solving linear inverse problems if the forward operator is given indirectly but specified via some input-output training pairs. We extend the approach in "Data driven regularization…
We asymptotically derive a non-linear Langevin-like equation with non-Gaussian white noise for a wide class of stochastic systems associated with multiple stochastic environments, by developing the expansion method in our previous paper [K.…
Logit regularization, the addition of a convex penalty directly in logit space, is widely used in modern classifiers, with label smoothing as a prominent example. While such methods often improve calibration and generalization, their…
We present the reduction of generalized Langevin equations to a coordinate-only stochastic model, which in its exact form, involves a forcing term with memory and a general Gaussian noise. It will be shown that a similar…
A continuous approximation framework for non-linear stochastic as well as deterministic discrete maps is developed. For the stochastic map with uncorelated Gaussian noise, by successively applying the It\^o lemma, we obtain a Langevin type…
The stochastic gradient Langevin Dynamics is one of the most fundamental algorithms to solve sampling problems and non-convex optimization appearing in several machine learning applications. Especially, its variance reduced versions have…
Gradient descent can be surprisingly good at optimizing deep neural networks without overfitting and without explicit regularization. We find that the discrete steps of gradient descent implicitly regularize models by penalizing gradient…
A random multiplicative process with additive noise is described by a Langevin equation. We show that the fluctuation-dissipation relation is satisfied in the Langevin model, if the noise strength is not so strong.
As an example of the nonlinear Fokker-Planck equation, the mean field Langevin dynamics recently attracts attention due to its connection to (noisy) gradient descent on infinitely wide neural networks in the mean field regime, and hence the…
We consider an evolution operator for a discrete Langevin equation with a strongly hyperbolic classical dynamics and noise with finite moments. Using a perturbative expansion of the evolution operator we calculate high order corrections to…
This article is concerned with sampling from Gibbs distributions $\pi(x)\propto e^{-U(x)}$ using Markov chain Monte Carlo methods. In particular, we investigate Langevin dynamics in the continuous- and the discrete-time setting for such…
Stochastic gradient descent (SGD) is a pillar of modern machine learning, serving as the go-to optimization algorithm for a diverse array of problems. While the empirical success of SGD is often attributed to its computational efficiency…
Stochastic Gradient Langevin Dynamics (SGLD) is a powerful algorithm for optimizing a non-convex objective, where a controlled and properly scaled Gaussian noise is added to the stochastic gradients to steer the iterates towards a global…
Convex optimization with sparsity-promoting convex regularization is a standard approach for estimating sparse signals in noise. In order to promote sparsity more strongly than convex regularization, it is also standard practice to employ…
Stochastic differential equations of Langevin-diffusion form have received significant attention, thanks to their foundational role in both Bayesian sampling algorithms and optimization in machine learning. In the latter, they serve as a…
In this paper, we consider the generalised (higher order) Langevin equation for the purpose of simulated annealing and optimisation of nonconvex functions. Our approach modifies the underdamped Langevin equation by replacing the Brownian…
We consider the problem of approximating the Langevin dynamics of inertial particles being transported by a background flow. In particular, we study an acceleration corrected advection-diffusion approximation to the Langevin dynamics, a…
A model has two main aims: predicting the behavior of a physical system and understanding its nature, that is how it works, at some desired level of abstraction. A promising recent approach to model building consists in deriving a…
The Langevin equation with multiplicative noise and state-dependent transport coefficient has to be always complemented with the proper interpretation rule of the noise, such as the Ito and Stratonovich conventions. Although the…
The purpose of this paper is to examine the Lagrangian stochastic modeling of the fluid velocity seen by inertial particles in a nonhomogeneous turbulent flow. A new Langevin-type model, compatible with the transport equation of the drift…