相关论文: Multiscale Gentlest Ascent Dynamics for Saddle Poi…

In this paper we propose a primal-dual dynamical approach to the minimization of a structured convex function consisting of a smooth term, a nonsmooth term, and the composition of another nonsmooth term with a linear continuous operator. In…

We present a novel approach to investigate the long-time stochastic dynamics of multi-dimensional classical systems, in contact with a heat-bath. When the potential energy landscape is rugged, the kinetics displays a decoupling of short and…

In this paper a spatial homogeneous vehicular traffic flow model based on a stochastic master equation of Boltzmann type in the acceleration variable is solved numerically for a special driver interaction model. The solution is done by a…

Diverse complex dynamical systems are known to exhibit abrupt regime shifts at bifurcation points of the saddle-node type. The dynamics of most of these systems, however, have a stochastic component resulting in noise driven regime shifts…

Stochastic gradient descent (SGD) is a standard optimization method to minimize a training error with respect to network parameters in modern neural network learning. However, it typically suffers from proliferation of saddle points in the…

This paper studies the projected saddle-point dynamics associated to a convex-concave function, which we term saddle function. The dynamics consists of gradient descent of the saddle function in variables corresponding to convexity and…

Stochastic gradient descent is a canonical tool for addressing stochastic optimization problems, and forms the bedrock of modern machine learning and statistics. In this work, we seek to balance the fact that attenuating step-size is…

In this paper we propose stochastic gradient-free methods and accelerated methods with momentum for solving stochastic optimization problems. All these methods rely on stochastic directions rather than stochastic gradients. We analyze the…

Convergence detection of iterative stochastic optimization methods is of great practical interest. This paper considers stochastic gradient descent (SGD) with a constant learning rate and momentum. We show that there exists a transient…

Traditional models of wormlike chains in shear flows at finite temperature approximate the equation of motion via finite difference discretization (bead and rod models). We introduce here a new method based on a spectral representation in…

In this paper, we study the minimax optimization problem in the smooth and strongly convex-strongly concave setting when we have access to noisy estimates of gradients. In particular, we first analyze the stochastic Gradient Descent Ascent…

The dynamics of systems of many degrees of freedom evolving on multiple scales are often modeled in terms of stochastic differential equations. Usually the structural form of these equations is unknown and the only manifestation of the…

Many interfacial phenomena in physical and biological systems are dominated by high order geometric quantities such as curvature. Here a semi-implicit method is combined with a level set jet scheme to handle stiff nonlinear advection…

We investigate the test risk of continuous-time stochastic gradient flow dynamics in learning theory. Using a path integral formulation we provide, in the regime of a small learning rate, a general formula for computing the difference…

We analyze the convergence rate of various momentum-based optimization algorithms from a dynamical systems point of view. Our analysis exploits fundamental topological properties, such as the continuous dependence of iterates on their…

Optimal processes in stochastic thermodynamics are a frontier for understanding the control and design of non-equilibrium systems, with broad practical applications in biology, chemistry, and nanoscale/mesoscale systems. Optimal mass…

This paper presents a rigorous numerical framework for computing multiple solutions of semilinear elliptic problems by spatiotemporal high-index saddle dynamics (HiSD), which extends the traditional HiSD to the continuous-in-space setting,…

In non-smooth stochastic optimization, we establish the non-convergence of the stochastic subgradient descent (SGD) to the critical points recently called active strict saddles by Davis and Drusvyatskiy. Such points lie on a manifold $M$…

A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…

We consider the problem of minimizing a convex function that is evolving according to unknown and possibly stochastic dynamics, which may depend jointly on time and on the decision variable itself. Such problems abound in the machine…