Related papers: On the implicit regularization of Langevin dynamic…
In overparametrized models, the noise in stochastic gradient descent (SGD) implicitly regularizes the optimization trajectory and determines which local minimum SGD converges to. Motivated by empirical studies that demonstrate that training…
We propose a novel approach to analyze generalization error for discretizations of Langevin diffusion, such as the stochastic gradient Langevin dynamics (SGLD). For an $\epsilon$ tolerance of expected generalization error, it is known that…
Langevin algorithms are gradient descent methods augmented with additive noise, and are widely used in Markov Chain Monte Carlo (MCMC) sampling, optimization, and machine learning. In recent years, the non-asymptotic analysis of Langevin…
Using the recently developed covariant Ito-Langevin dynamics, we develop a non-equilibrium thermodynamic theory for small systems coupled to multiplicative noises. The theory is based on Ito-calculus, and is fully covariant under…
Diffusion/score-based models have emerged as powerful generative models, capable of generating high-quality samples that mimic the training data distribution. However, it has been observed that they are prone to reproducing training…
We use the perturbative renormalization group to study classical stochastic processes with memory. We focus on the generalized Langevin dynamics of the \phi^4 Ginzburg-Landau model with additive noise, the correlations of which are local in…
We present a derivation of a coarse-grained model from the Langevin dynamics. The focus is placed on the memory kernel function and the fluctuation-dissipation theorem. Also presented is an hierarchy of approximations for the memory and…
Neural populations exhibit latent dynamical structures that drive time-evolving spiking activities, motivating the search for models that capture both intrinsic network dynamics and external unobserved influences. In this work, we introduce…
We consider equations of nonlinear transport on the circle with regular self interactions appearing in aggregation models and deterministic mean field dynamics. We introduce a random perturbation of such systems through a stochastic…
In this paper we consider a new probability sampling methods based on Langevin diffusion dynamics to resolve the problem of existing Monte Carlo algorithms when draw samples from high dimensional target densities. We extent…
How can we learn the laws underlying the dynamics of stochastic systems when their trajectories are sampled sparsely in time? Existing methods either require temporally resolved high-frequency observations, or rely on geometric arguments…
The advantages of performing Langevin Dynamics in extended systems are discussed. A simple Langevin Dynamics scheme for producing the canonical ensemble is reviewed, and is then extended to the Hoover ensemble. We show that the resulting…
Implicit inverse problems, in which noisy observations of a physical quantity are used to infer a nonlinear functional applied to an associated function, are inherently ill posed and often exhibit non uniqueness of solutions. Such problems…
We introduce a novel approach for decomposing and learning every scale of a given multiscale objective function in $\mathbb{R}^d$, where $d\ge 1$. This approach leverages a recently demonstrated implicit bias of the optimization method of…
Errors dynamics captures the evolution of the state errors between two distinct trajectories, that are governed by the same system rule but initiated or perturbed differently. In particular, state observer error dynamics analysis in matrix…
Continuous-time models provide important insights into the training dynamics of optimization algorithms in deep learning. In this work, we establish a non-asymptotic convergence analysis of stochastic gradient Langevin dynamics (SGLD),…
Langevin simulation provides an effective way to study collisional effects in beams by reducing the six-dimensional Fokker-Planck equation to a group of stochastic ordinary differential equations. These resulting equations usually have…
In a system of noisy self-propelled particles with interactions that favor directional alignment, collective motion will appear if the density of particles is beyond a critical density. Starting with a reduced model for collective motion,…
Implicit regularization refers to the tendency of local search algorithms to converge to low-dimensional solutions, even when such structures are not explicitly enforced. Despite its ubiquity, the mechanism underlying this behavior remains…
We interpret the variational inference of the Stochastic Gradient Descent (SGD) as minimizing a new potential function named the \textit{quasi-potential}. We analytically construct the quasi-potential function in the case when the loss…