Related papers: Stability of first-order methods in tame optimizat…
We consider linear dynamical systems consisting of ordinary differential equations with high dimensionality. The aim of model order reduction is to construct an approximating system of a much lower dimension. Therein, the reduced system may…
The signal differentiation problem involves the development of algorithms that allow to recover a signal's derivatives from noisy measurements. This paper develops a first-order differentiator with the following combination of properties:…
The analysis on the global stability of Riemannian gradient descent method in manifold optimization (i.e., it avoids strict saddle points for almost all initializations) due to Lee et al. (Math. Program. 176:311-337) is corrected. Moreover,…
Many recent studies on first-order methods (FOMs) focus on \emph{composite non-convex non-smooth} optimization with linear and/or nonlinear function constraints. Upper (or worst-case) complexity bounds have been established for these…
Decentralized optimization to minimize a finite sum of functions over a network of nodes has been a significant focus within control and signal processing research due to its natural relevance to optimal control and signal estimation…
Recently, many machine learning optimizers have been analysed considering them as the asymptotic limit of some differential equations when the step size goes to zero. In other words, the optimizers can be seen as a finite difference scheme…
In this paper, we propose a systematic approach for extending first-order optimization algorithms, originally designed for unconstrained strongly convex problems, to handle closed and convex set constraints. We show that the resulting…
We establish that first-order methods avoid saddle points for almost all initializations. Our results apply to a wide variety of first-order methods, including gradient descent, block coordinate descent, mirror descent and variants thereof.…
In this paper, we give explicit estimates that insure the existence of solutions for first order partial differential operators on compact manifolds, using a viscosity method. In the linear case, an explicit integral formula can be found,…
In this paper, we present several new results on minimizing a nonsmooth and nonconvex function under a Lipschitz condition. Recent work shows that while the classical notion of Clarke stationarity is computationally intractable up to some…
We develop a theory of accelerated first-order optimization from the viewpoint of differential equations and Lyapunov functions. Building upon the previous work of many researchers, we consider differential equations which model the…
We derive several numerical methods for designing optimized first-order algorithms in unconstrained convex optimization settings. Our methods are based on the Performance Estimation Problem (PEP) framework, which casts the worst-case…
In this paper we consider discrete gradient methods for approximating the solution and preserving a first integral (also called a constant of motion) of autonomous ordinary differential equations. We prove under mild conditions for a large…
The stable principal component pursuit (SPCP) problem is a non-smooth convex optimization problem, the solution of which has been shown both in theory and in practice to enable one to recover the low rank and sparse components of a matrix…
This work establishes a rigorous connection between stability properties of discrete-time algorithms (DTAs) and corresponding continuous-time dynamical systems derived through $ O(s^r) $-resolution ordinary differential equations (ODEs). We…
In many important machine learning applications, the standard assumption of having a globally Lipschitz continuous gradient may fail to hold. This paper delves into a more general $(L_0, L_1)$-smoothness setting, which gains particular…
We analyze worst-case convergence guarantees of first-order optimization methods over a function class extending that of smooth and convex functions. This class contains convex functions that admit a simple quadratic upper bound. Its study…
We show that any solution of a smooth first order dynamic equation can be made Lyapunov stable at will by the choice of an appropriate time-dependent Riemannian metric.
We consider the stability of Robust Optimization problems with respect to perturbations in their uncertainty sets. We focus on Linear Optimization problems, including those with a possibly infinite number of constraints, also known as…
The derivation of second-order ordinary differential equations (ODEs) as continuous-time limits of optimization algorithms has been shown to be an effective tool for the analysis of these algorithms. Additionally, discretizing…