Related papers: On the implicit regularization of Langevin dynamic…
We study the problem of sampling from a probability distribution on $\mathbb R^p$ defined via a convex and smooth potential function. We consider a continuous-time diffusion-type process, termed Penalized Langevin dynamics (PLD), the drift…
It is important to understand how dropout, a popular regularization method, aids in achieving a good generalization solution during neural network training. In this work, we present a theoretical derivation of an implicit regularization of…
The behavior of the self diffusion constant of Langevin particles interacting via a pairwise interaction is considered. The diffusion constant is calculated approximately within a perturbation theory in the potential strength about the bare…
In this paper, we study the implicit regularization of stochastic gradient descent (SGD) through the lens of {\em dynamical stability} (Wu et al., 2018). We start by revising existing stability analyses of SGD, showing how the Frobenius…
Backward diffusion is a prototype ill-posed evolution: high spatial frequencies grow exponentially in time, so mesh-based time marching without explicit regularization is quickly overwhelmed by noise. Schr\"odingerization embeds the…
Inverse problems and regularization theory is a central theme in contemporary signal processing, where the goal is to reconstruct an unknown signal from partial indirect, and possibly noisy, measurements of it. A now standard method for…
Large learning rates, when applied to gradient descent for nonconvex optimization, yield various implicit biases including the edge of stability (Cohen et al., 2021), balancing (Wang et al., 2022), and catapult (Lewkowycz et al., 2020).…
Continuous time random walks and Langevin equations are two classes of stochastic models for describing the dynamics of particles in the natural world. While some of the processes can be conveniently characterized by both of them, more…
In this note, we consider the construction of a one-dimensional stable Langevin type process confined in the upper half-plane and submitted to reflective-diffusive boundary conditions whenever the particle position hits 0. We show that two…
The inverse Langevin function is a fundamental part of the statistical chain models used to describe the behavior of polymeric-like materials, appearing also in other fields such as magnetism, molecular dynamics and even biomechanics. In…
We demonstrate that the conventional path integral formulations generate inconsistent results exemplified by the geometric Brownian motion under the general stochastic interpretation. We thus develop a novel path integral formulation for…
We study how thermal fluctuations affect the dynamics of vortices in the two-dimensional classical, ferromagnetic, anisotropic Heisenberg model depending on their additive or multiplicative character. Using a collective coordinate theory,…
We discuss how to derive a Langevin equation (LE) in non standard systems, i.e. when the kinetic part of the Hamiltonian is not the usual quadratic function. This generalization allows to consider also cases with negative absolute…
Non-spherical particles transported by an anisotropic turbulent flow preferentially align with the mean shear and intermittently tumble when the local strain fluctuates. Such an intricate behaviour is here studied for inertialess,…
Fluctuations and noise may alter the behavior of dynamical systems considerably. For example, oscillations may be sustained by demographic fluctuations in biological systems where a stable fixed point is found in the absence of noise. We…
We consider the problem of sampling from a target distribution, which is \emph {not necessarily logconcave}, in the context of empirical risk minimization and stochastic optimization as presented in Raginsky et al. (2017). Non-asymptotic…
For the Langevin model of the dynamics of a Brownian particle with perturbations orthogonal to its current velocity, in a regime when the particle velocity modulus becomes constant, an equation for the characteristic function $\psi…
Connected with the rise of interest in inverse problems is the development and analysis of regularization methods, which are a necessity due to the ill-posedness of inverse problems. Tikhonov-type regularization methods are very popular in…
We study the problem of non-convex optimization using Stochastic Gradient Langevin Dynamics (SGLD). SGLD is a natural and popular variation of stochastic gradient descent where at each step, appropriately scaled Gaussian noise is added. To…
Injecting noise within gradient descent has several desirable features, such as smoothing and regularizing properties. In this paper, we investigate the effects of injecting noise before computing a gradient step. We demonstrate that small…