相关论文: Study of micro-macro acceleration schemes for line…

Stochastic gradient methods enable learning probabilistic models from large amounts of data. While large step-sizes (learning rates) have shown to be best for least-squares (e.g., Gaussian noise) once combined with parameter averaging,…

We present an ``equation-free'' multiscale approach to the simulation of unsteady diffusion in a random medium. The diffusivity of the medium is modeled as a random field with short correlation length, and the governing equations are cast…

We consider the minimization of an objective function given access to unbiased estimates of its gradient through stochastic gradient descent (SGD) with constant step-size. While the detailed analysis was only performed for quadratic…

Stochastic gradient methods are scalable for solving large-scale optimization problems that involve empirical expectations of loss functions. Existing results mainly apply to optimization problems where the objectives are one- or two-level…

This work is concerned with the dynamics of a class of slow-fast stochastic dynamical systems with non-Gaussian stable L\'evy noise with a scale parameter. Slow manifolds with exponentially tracking property are constructed, eliminating the…

Owing to their stability and convergence speed, extragradient methods have become a staple for solving large-scale saddle-point problems in machine learning. The basic premise of these algorithms is the use of an extrapolation step before…

In the presence of multiscale dynamics in a reaction network, direct simulation methods become inefficient as they can only advance the system on the smallest scale. This work presents stochastic averaging techniques to accelerate…

In this paper, we propose a novel technique to implement stochastic gradient methods, which are beneficial for learning from large datasets, through accelerated stochastic dynamics. A stochastic gradient method is based on mini-batch…

We extend the Approximate-Proximal Point (aProx) family of model-based methods for solving stochastic convex optimization problems, including stochastic subgradient, proximal point, and bundle methods, to the minibatch and accelerated…

Extrapolation is a well-known technique for solving convex optimization and variational inequalities and recently attracts some attention for non-convex optimization. Several recent works have empirically shown its success in some machine…

This paper is concerned with fully discrete mixed finite element approximations of the time-dependent stochastic Stokes equations with multiplicative noise. A prototypical method, which comprises of the Euler-Maruyama scheme for time…

When simulating multiscale stochastic differential equations (SDEs) in high-dimensions, separation of timescales, stochastic noise and high-dimensionality can make simulations prohibitively expensive. The computational cost is dictated by…

Deep learning networks are typically trained by Stochastic Gradient Descent (SGD) methods that iteratively improve the model parameters by estimating a gradient on a very small fraction of the training data. A major roadblock faced when…

Multiple time scale stochastic dynamical systems are ubiquitous in science and engineering, and the reduction of such systems and their models to only their slow components is often essential for scientific computation and further analysis.…

This work develops Monte Carlo Euler adaptive time stepping methods for the weak approximation problem of jump diffusion driven stochastic differential equations. The main result is the derivation of a new expansion for the omputational…

We introduce a class of unconditionally energy stable, high order accurate schemes for gradient flows in a very general setting. The new schemes are a high order analogue of the minimizing movements approach for generating a time discrete…

A computational tool for coarse-graining nonlinear systems of ordinary differential equations in time is discussed. Three illustrative model examples are worked out that demonstrate the range of capability of the method. This includes the…

This paper addresses the design and analysis of a multivariable gradient-based stochastic extremum-seeking control method for multi-input systems with arbitrary input delays. The approach accommodates systems with distinct time delays…

In this paper, we proved moderate deviation principles for a fully coupled two-time-scale stochastic systems, where the slow process is given by stochastic differential equations with small noise, while the fast process is a rapidly…

We consider systems of stochastic differential equations with multiple scales and small noise and assume that the coefficients of the equations are ergodic and stationary random fields. Our goal is to construct provably-efficient importance…