Related papers: SMD-based numerical stochastic perturbation theory
Stochastic Gradient Langevin Dynamics (SGLD) is a powerful algorithm for optimizing a non-convex objective, where a controlled and properly scaled Gaussian noise is added to the stochastic gradients to steer the iterates towards a global…
We derive and analyze numerical methods for underdamped (kinetic) Langevin dynamics in a domain with elastic reflection at the boundary. First-order approximations are based on an Euler-type scheme incorporating collision-handling at the…
Langevin simulation provides an effective way to study collisional effects in beams by reducing the six-dimensional Fokker-Planck equation to a group of stochastic ordinary differential equations. These resulting equations usually have…
Formulated is a new systematic method for obtaining higher order corrections in numerical simulation of stochastic differential equations (SDEs), i.e., Langevin equations. Random walk step algorithms within a given order of finite $\Delta…
The stochastic gradient Langevin Dynamics is one of the most fundamental algorithms to solve sampling problems and non-convex optimization appearing in several machine learning applications. Especially, its variance reduced versions have…
Numerical Stochastic Perturbation Theory (NSPT) has over the years proved to be a valuable tool, in particular being able to reach unprecedented orders for Lattice Gauge Theories, whose perturbative expansions are notoriously cumbersome.…
We study stochastic gradient descent (SGD) for composite optimization problems with $N$ sequential operators subject to perturbations in both the forward and backward passes. Unlike classical analyses that treat gradient noise as additive…
Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and…
We study the mixing properties of an important optimization algorithm of machine learning: the stochastic gradient Langevin dynamics (SGLD) with a fixed step size. The data stream is not assumed to be independent hence the SGLD is not a…
Stochastic Gradient Langevin Dynamics (SGLD) is a sampling scheme for Bayesian modeling adapted to large datasets and models. SGLD relies on the injection of Gaussian Noise at each step of a Stochastic Gradient Descent (SGD) update. In this…
Inverse problems are paramount in Science and Engineering. In this paper, we consider the setup of Statistical Inverse Problem (SIP) and demonstrate how Stochastic Gradient Descent (SGD) algorithms can be used in the linear SIP setting. We…
One approach for reducing run time and improving efficiency of machine learning is to reduce the convergence rate of the optimization algorithm used. Shuffling is an algorithm technique that is widely used in machine learning, but it only…
Many relevant problems in the area of systems and control, such as controller synthesis, observer design and model reduction, can be viewed as optimization problems involving dynamical systems: for instance, maximizing performance in the…
We study the Stochastic Gradient Langevin Dynamics (SGLD) algorithm for non-convex optimization. The algorithm performs stochastic gradient descent, where in each step it injects appropriately scaled Gaussian noise to the update. We analyze…
A framework is introduced for solving a sequence of slowly changing optimization problems, including those arising in regression and classification applications, using optimization algorithms such as stochastic gradient descent (SGD). The…
We develope a perturbation theory for stochastic differential equations (SDEs) by which we mean both stochastic ordinary differential equations (SODEs) and stochastic partial differential equations (SPDEs). In particular, we estimate the $…
In this paper we present a perturbative procedure that allows one to numerically solve diffusive non-Markovian Stochastic Schr\"odinger equations, for a wide range of memory functions. To illustrate this procedure numerical results are…
Stochastic gradient descent with momentum (SGDm) is one of the most popular optimization algorithms in deep learning. While there is a rich theory of SGDm for convex problems, the theory is considerably less developed in the context of deep…
The explicit semiclassical treatment of the logarithmic perturbation theory for the bound-state problem for the spherical anharmonic oscillator and the screened Coulomb potential is developed. Based upon the $\hbar$-expansions and suitable…
We introduce a new approach to develop stochastic optimization algorithms for a class of stochastic composite and possibly nonconvex optimization problems. The main idea is to combine two stochastic estimators to create a new hybrid one. We…