Related papers: On the Instability of Nesterov's ODE under Non-Con…
We study sufficient conditions for stability and recurrence in a class of singularly perturbed stochastic hybrid dynamical systems. The systems considered combine multi-time-scale deterministic continuous-time dynamics, modeled by…
Vector fields that are discontinuous on codimension-one surfaces are known as Filippov systems and can have attracting periodic orbits involving segments that are contained on a discontinuity surface of the vector field. In this paper we…
Much recent interest has focused on the design of optimization algorithms from the discretization of an associated optimization flow, i.e., a system of differential equations (ODEs) whose trajectories solve an associated optimization…
While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems. Based on a proximal primal-dual approach, this paper…
In this paper we deal with a general second order continuous dynamical system associated to a convex minimization problem with a Fr\`echet differentiable objective function. We show that inertial algorithms, such as Nesterov's algorithm,…
In convex optimization, continuous-time counterparts have been a fruitful tool for analyzing momentum algorithms. Fewer such examples are available when the function to minimize is non-convex. In several cases, discrepancies arise between…
This paper considers a class of reinforcement-learning that belongs to the family of Learning Automata and provides a stochastic-stability analysis in strategic-form games. For this class of dynamics, convergence to pure Nash equilibria has…
Ever since the original algorithm by Nesterov (1983), the true nature of the acceleration phenomenon has remained elusive, with various interpretations of why the method is actually faster. The diagnosis of the algorithm through the lens of…
Stochastic approximation is a class of algorithms that update a vector iteratively, incrementally, and stochastically, including, e.g., stochastic gradient descent and temporal difference learning. One fundamental challenge in analyzing a…
We show that Nesterov acceleration is an optimal-order iterative regularization method for linear ill-posed problems provided that a parameter is chosen accordingly to the smoothness of the solution. This result is proven both for an a…
The paper describes a novel method for studying the stability of nonautonomous dynamical systems. This method based on the flow and divergence of the vector field with coupling to the method of Lyapunov functions. The necessary and…
Arguably, the two most popular accelerated or momentum-based optimization methods in machine learning are Nesterov's accelerated gradient and Polyaks's heavy ball, both corresponding to different discretizations of a particular second order…
This paper is concerned with convergence of stochastic gradient algorithms with momentum terms in the nonconvex setting. A class of stochastic momentum methods, including stochastic gradient descent, heavy ball, and Nesterov's accelerated…
We propose and analyze a class of second-order dynamical systems for continuous-time optimization that incorporate fractional-order gradient terms. The system is given by \begin{equation} \ddot{x}(t) + \frac{\alpha}{t}\dot{x}(t) +…
We study the algorithmic stability of Nesterov's accelerated gradient method. For convex quadratic objectives, Chen et al. (2018) proved that the uniform stability of the method grows quadratically with the number of optimization steps, and…
A non-perturbative approach to the time-averaging of nonlinear, autonomous ODE systems is developed based on invariant manifold methodology. The method is implemented computationally and applied to model problems arising in the mechanics of…
We prove the global stability of the Minkowski space viewed as the trivial solution of the Einstein-Vlasov system. To estimate the Vlasov field, we use the vector field and modified vector field techniques developed in [FJS15; FJS17]. In…
We propose an adaptive smoothing algorithm based on Nesterov's smoothing technique in \cite{Nesterov2005c} for solving "fully" nonsmooth composite convex optimization problems. Our method combines both Nesterov's accelerated proximal…
Frequency responses of multi-degree-of-freedom mechanical systems with weak forcing and damping can be studied as perturbations from their conservative limit. Specifically, recent results show how bifurcations near resonances can be…
For first-order optimization of non-convex functions with Lipschitz continuous gradient and Hessian, the best known complexity for reaching an $\varepsilon$-approximation of a stationary point is $\mathcal{O}(\varepsilon^{-7/4})$. Existing…