Related papers: A Speed Restart Scheme for a Dynamical System with…

First-order methods with momentum such as Nesterov's fast gradient method are very useful for convex optimization problems, but can exhibit undesirable oscillations yielding slow convergence rates for some applications. An adaptive…

This paper develops a primal-dual dynamical system where the coefficients are designed in closed-loop way for solving a convex optimization problem with linear equality constraints. We first introduce a ``second-order primal" +…

This paper is devoted to the study of an inertial accelerated primal-dual algorithm, which is based on a second-order differential system with time scaling, for solving a non-smooth convex optimization problem with linear equality…

We provide a simple and generic adaptive restart scheme for convex optimization that is able to achieve worst-case bounds matching (up to constant multiplicative factors) optimal restart schemes that require knowledge of problem specific…

Fast convergence and high-quality image recovery are two essential features of algorithms for solving ill-posed imaging inverse problems. Existing methods, such as regularization by denoising (RED), often focus on designing sophisticated…

We study nonsmooth convex minimization through a continuous-time dynamical system that can be seen as a high-resolution ODE of Nesterov Accelerated Gradient (NAG) adapted to the nonsmooth case. We apply a time-varying Moreau envelope…

In a Hilbert space setting, for convex optimization, we show the convergence of the iterates to optimal solutions for a class of accelerated first-order algorithms. They can be interpreted as discrete temporal versions of an inertial…

It is known that the restarted full orthogonalization method (FOM) outperforms the restarted generalized minimum residual (GMRES) method in several circumstances for solving shifted linear systems when the shifts are handled simultaneously.…

We propose an optimization algorithm called Frictionless Hamiltonian Descent, which is a direct counterpart of classical Hamiltonian Monte Carlo in sampling. We analyze Frictionless Hamiltonian Descent for strongly convex quadratic…

We first study the fast minimization properties of the trajectories of the second-order evolution equation $$\ddot{x}(t) + \frac{\alpha}{t} \dot{x}(t) + \beta \nabla^2 \Phi (x(t))\dot{x} (t) + \nabla \Phi (x(t)) = 0,$$ where $\Phi:\mathcal…

How long does a trajectory take to reach a stable equilibrium point in the basin of attraction of a dynamical system? This is a question of quite general interest, and has stimulated a lot of activities in dynamical and stochastic systems…

In this paper we study a second order dynamical system with variable coefficients in connection to the minimization problem of a smooth nonconvex function. The convergence of the trajectories generated by the dynamical system to a critical…

Restart techniques are common in gradient-free optimization to deal with multimodal functions. Partial warm restarts are also gaining popularity in gradient-based optimization to improve the rate of convergence in accelerated gradient…

First-order optimization algorithms can be considered as a discretization of ordinary differential equations (ODEs) \cite{su2014differential}. In this perspective, studying the properties of the corresponding trajectories may lead to…

We analyze the convergence rate of various momentum-based optimization algorithms from a dynamical systems point of view. Our analysis exploits fundamental topological properties, such as the continuous dependence of iterates on their…

A direct data-driven iterative algorithm is developed to accurately estimate the $H_\infty$ norm of a linear time-invariant system from continuous operation, i.e., without resetting the system. The main technical step involves a…

Various types of parameter restart schemes have been proposed for accelerated gradient algorithms to facilitate their practical convergence in convex optimization. However, the convergence properties of accelerated gradient algorithms under…

In this paper, we consider diagonal hyperbolic systems with monotone continuous initial data. We propose a natural semi-explicit and upwind first order scheme. Under a certain non-negativity condition on the Jacobian matrix of the…

We propose a third order dynamical system for solving a nonlinear equation in Hilbert spaces where the operator is cocoercive with respect to the solutions set. Under mild conditions on the parameters, we establish the existence and…

We propose a structure-adaptive variant of the state-of-the-art stochastic variance-reduced gradient algorithm Katyusha for regularized empirical risk minimization. The proposed method is able to exploit the intrinsic low-dimensional…