Related papers: Continuous Newton-like Methods featuring Inertia a…
We consider the dynamical system \begin{equation*}\left\{ \begin{array}{ll} v(t)\in\partial\phi(x(t))\\ \lambda\dot x(t) + \dot v(t) + v(t) + \nabla \psi(x(t))=0, \end{array}\right.\end{equation*} where $\phi:\R^n\to\R\cup\{+\infty\}$ is a…
In this paper, we propose in a Hilbertian setting a second-order time-continuous dynamic system with fast convergence guarantees to solve structured convex minimization problems with an affine constraint. The system is associated with the…
Optimization problems emerging in most of the real-world applications are dynamic, where either the objective function or the constraints change continuously over time. This paper proposes projected primal-dual dynamical system approaches…
A dynamical system is defined in terms of the gradient of a payoff function. Dynamical variables are of two types, ascent and descent. The ascent variables move in the direction of the gradient, while the descent variables move in the…
We will consider the damped Newton method for strongly monotone and Lipschitz continuous operator equations in a variational setting. We will provide a very accessible justification why the undamped Newton method performs better than its…
We propose a continuous-time scheme for large-scale optimization that introduces individual, adaptive momentum coefficients regulated by the kinetic energy of each model parameter. This approach automatically adjusts to local landscape…
Quasi-Newton methods are widely used in practise for convex loss minimization problems. These methods exhibit good empirical performance on a wide variety of tasks and enjoy super-linear convergence to the optimal solution. For large-scale…
A new class of projected dynamical systems of third order is investigated for quasi (parametric) variational inequalities in which the convex set in the classical variational inequality also depends upon the solution explicitly or…
We propose a novel continuous-time algorithm for inequality-constrained convex optimization inspired by proportional-integral control. Unlike the popular primal-dual gradient dynamics, our method includes a proportional term to control the…
In this paper, we derive an accelerated continuous-time formulation of Adam by modeling it as a second-order integro-differential dynamical system. We relate this inertial nonlocal model to an existing first-order nonlocal Adam flow through…
Aiming to provide a new class of game dynamics with good long-term rationality properties, we derive a second-order inertial system that builds on the widely studied "heavy ball with friction" optimization method. By exploiting a well-known…
A new variant of Newton's method for empirical risk minimization is studied, where at each iteration of the optimization algorithm, the gradient and Hessian of the objective function are replaced by robust estimators taken from existing…
We show that accelerated gradient descent, averaged gradient descent and the heavy-ball method for non-strongly-convex problems may be reformulated as constant parameter second-order difference equation algorithms, where stability of the…
In a real Hilbert space, we consider two classical problems: the global minimization of a smooth and convex function $f$ (i.e., a convex optimization problem) and finding the zeros of a monotone and continuous operator $V$ (i.e., a monotone…
In this paper, by combining the algorithm New Q-Newton's method - developed in previous joint work of the author - with Armijo's Backtracking line search, we resolve convergence issues encountered by Newton's method (e.g. convergence to a…
By time discretization of a second-order primal-dual dynamical system with damping $\alpha/t$ where an inertial construction in the sense of Nesterov is needed only for the primal variable, we propose a fast primal-dual algorithm for a…
In this note we approach the classical, Newtonian, gravitational $N$-body problem by mean of a new, original numerical integration method. After a short summary of the fundamental characteristics of the problem, including a sketch of some…
In this paper, we present a novel Newton-based extremum seeking controller for the solution of multivariable model-free optimization problems in static maps. Unlike existing asymptotic and fixed-time results in the literature, we present a…
In this article, we provide a novel and broadly-applicable contraction-theoretic approach to continuous-time time-varying convex optimization. For any parameter-dependent contracting dynamics, we show that the tracking error is…
We examine a class of stochastic differential inclusions involving multiscale effects designed to solve a class of generalized variational inequalities. This class of problems contains constrained convex non-smooth optimization problems,…