English

Nesterov's acceleration and Polyak's heavy ball method in continuous time: convergence rate analysis under geometric conditions and perturbations

Optimization and Control 2019-07-08 v1

Abstract

In this article a family of second order ODEs associated to inertial gradient descend is studied. These ODEs are widely used to build trajectories converging to a minimizer xx^* of a function FF, possibly convex. This family includes the continuous version of the Nesterov inertial scheme and the continuous heavy ball method. Several damping parameters, not necessarily vanishing, and a perturbation term gg are thus considered. The damping parameter is linked to the inertia of the associated inertial scheme and the perturbation term gg is linked to the error that can be done on the gradient of the function FF. This article presents new asymptotic bounds on F(x(t))F(x)F(x(t))-F(x^*) where xx is a solution of the ODE, when FF is convex and satisfies local geometrical properties such as {\L}ojasiewicz properties and under integrability conditions on gg. Even if geometrical properties and perturbations were already studied for most ODEs of these families, it is the first time they are jointly studied. All these results give an insight on the behavior of these inertial and perturbed algorithms if FF satisfies some {\L}ojasiewicz properties especially in the setting of stochastic algorithms.

Keywords

Cite

@article{arxiv.1907.02710,
  title  = {Nesterov's acceleration and Polyak's heavy ball method in continuous time: convergence rate analysis under geometric conditions and perturbations},
  author = {Othmane Sebbouh and Charles Dossal and Aude Rondepierre},
  journal= {arXiv preprint arXiv:1907.02710},
  year   = {2019}
}