Related papers: Lyapunov Functions for First-Order Methods: Tight …

We propose new continuous-time formulations for first-order stochastic optimization algorithms such as mini-batch gradient descent and variance-reduced methods. We exploit these continuous-time models, together with simple Lyapunov analysis…

We present a new approach for constructing polytope Lyapunov functions for continuous-time linear switching systems (LSS). This allows us to decide the stability of LSS and to compute the Lyapunov exponent with a good precision in…

We propose an acceleration scheme for first-order methods (FOMs) for convex quadratic programs (QPs) that is analogous to Anderson acceleration and the Generalized Minimal Residual algorithm for linear systems. We motivate our proposed…

This paper deals with a Tikhonov regularized second-order plus first-order primal-dual dynamical system with time scaling for separable convex optimization problems with linear equality constraints. This system consists of two second-order…

Control Lyapunov functions are traditionally used to design a controller which ensures convergence to a desired state, yet deriving these functions for nonlinear systems remains a complex challenge. This paper presents a novel,…

First-order optimization methods have attracted a lot of attention due to their practical success in many applications, including in machine learning. Obtaining convergence guarantees and worst-case performance certificates for first-order…

In this paper we consider stochastic composite convex optimization problems with the objective function satisfying a stochastic bounded gradient condition, with or without a quadratic functional growth property. These models include the…

Despite impressive results, reinforcement learning (RL) suffers from slow convergence and requires a large variety of tuning strategies. In this paper, we investigate the ability of RL algorithms on simple continuous control tasks. We show…

We present a kernel-based methodology for constructing Lyapunov functions for nonlinear dynamical systems using approximate Koopman eigenfunctions. Our approach decomposes principal Koopman eigenfunctions into linear and nonlinear…

Recent advances (Sherman, 2017; Sidford and Tian, 2018; Cohen et al., 2021) have overcome the fundamental barrier of dimension dependence in the iteration complexity of solving $\ell_\infty$ regression with first-order methods. Yet it…

Traditionally, there are several polynomial algorithms for linear programming including the ellipsoid method, the interior point method and other variants. Recently, Chubanov [Chubanov, 2015] proposed a projection and rescaling algorithm,…

Deep learning methods have been widely used in robotic applications, making learning-enabled control design for complex nonlinear systems a promising direction. Although deep reinforcement learning methods have demonstrated impressive…

This paper deals with an inertial proximal algorithm that contains a Tikhonov regularization term, in connection to the minimization problem of a convex lower semicontinuous function $f$. We show that for appropriate Tikhonov regularization…

In this paper, by using a characterization of functions having fractional derivative, we propose a rigorous fractional Lyapunov function candidate method to analyze stability of fractional-order nonlinear systems. First, we prove an…

We consider the constrained Linear Inverse Problem (LIP), where a certain atomic norm (like the $\ell_1 $ norm) is minimized subject to a quadratic constraint. Typically, such cost functions are non-differentiable, which makes them not…

A framework is developed for applying accelerated methods to general hyperbolic programming, including linear, second-order cone, and semidefinite programming as special cases. The approach replaces a hyperbolic program with a convex…

First-order methods have been popularly used for solving large-scale problems. However, many existing works only consider unconstrained problems or those with simple constraint. In this paper, we develop two first-order methods for…

In this paper, we conduct a theoretical and numerical study of the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) under strong convexity assumptions. We propose an autonomous Lyapunov function that reflects the strong convexity of…

This paper considers solving distributed optimization problems in peer-to-peer multi-agent networks. The network is synchronous and connected. By using the proportional-integral (PI) control strategy, various algorithms with fixed stepsize…

We propose a single time-scale stochastic subgradient method for constrained optimization of a composition of several nonsmooth and nonconvex functions. The functions are assumed to be locally Lipschitz and differentiable in a generalized…