Related papers: The connections between Lyapunov functions for som…

We present a coupled system of ODEs which, when discretized with a constant time step/learning rate, recovers Nesterov's accelerated gradient descent algorithm. The same ODEs, when discretized with a decreasing learning rate, leads to novel…

The generalized Lasso is a remarkably versatile and extensively utilized model across a broad spectrum of domains, including statistics, machine learning, and image science. Among the optimization techniques employed to address the…

In this paper, we propose a second-order continuous primal-dual dynamical system with time-dependent positive damping terms for a separable convex optimization problem with linear equality constraints. By the Lyapunov function approach, we…

This paper deals with a second order dynamical system with a Tikhonov regularization term in connection to the minimization problem of a convex Fr\'echet differentiable function. The fact that beside the asymptotically vanishing damping we…

We introduce an autonomous system with closed-loop damping for first-order convex optimization. While, to this day, optimal rates of convergence are almost exclusively achieved by non-autonomous methods via open-loop damping (e.g.,…

The paper studies numerical methods that preserve a Lyapunov function of a dynamical system, i.e. numerical approximations whose energy decreases, just like in the original differential equation. With this aim, a discrete gradient method is…

We develop a theory of accelerated first-order optimization from the viewpoint of differential equations and Lyapunov functions. Building upon the previous work of many researchers, we consider differential equations which model the…

This paper provides a self-contained ordinary differential equation solver approach for separable convex optimization problems. A novel primal-dual dynamical system with built-in time rescaling factors is introduced, and the exponential…

We present two classes of differentially private optimization algorithms derived from the well-known accelerated first-order methods. The first algorithm is inspired by Polyak's heavy ball method and employs a smoothing approach to decrease…

We focus on the solutions of second-order stable linear difference equations and demonstrate that their behavior can be non-monotone and exhibit peak effects depending on initial conditions. The results are applied to the analysis of the…

We propose a family of optimization methods that achieve linear convergence using first-order gradient information and constant step sizes on a class of convex functions much larger than the smooth and strongly convex ones. This larger…

We provide algorithms for computing a Lyapunov function for a class of systems where the state trajectories are constrained to evolve within a closed convex set. The dynamical systems that we consider comprise a differential equation which…

We study the convergence rate of a family of inertial algorithms, which can be obtained by discretization of an inertial system combining asymptotic vanishing viscous and Hessian-driven damping. We establish a fast sublinear convergence…

We study first-order optimization methods obtained by discretizing ordinary differential equations (ODEs) corresponding to Nesterov's accelerated gradient methods (NAGs) and Polyak's heavy-ball method. We consider three discretization…

Mirror descent plays a crucial role in constrained optimization and acceleration schemes, along with its corresponding low-resolution ordinary differential equations (ODEs) framework have been proposed. However, the low-resolution ODEs are…

We derive a second-order ordinary differential equation (ODE) which is the limit of Nesterov's accelerated gradient method. This ODE exhibits approximate equivalence to Nesterov's scheme and thus can serve as a tool for analysis. We show…

We analyze the convergence rate of a family of inertial algorithms, which can be obtained by discretization of an inertial system with Hessian-driven damping. We recover a convergence rate, up to a factor of 2 speedup upon Nesterov's…

We introduce a novel primal-dual flow for affine constrained convex optimization problems. As a modification of the standard saddle-point system, our primal-dual flow is proved to possess the exponential decay property, in terms of a…

We develop a distributed algorithm for convex Empirical Risk Minimization, the problem of minimizing large but finite sum of convex functions over networks. The proposed algorithm is derived from directly discretizing the second-order…

In this work, we investigate a second-order dynamical system with Hessian-driven damping tailored for a class of nonconvex functions called strongly quasiconvex. Buil\-ding upon this continuous-time model, we derive two discrete-time…