Related papers: An Abstract Lyapunov Control Optimizer: Local Stab…
In this work, the problem of optimizing damper positions in vibrational systems is investigated. The objective is to determine the positions of external dampers in such a way that the influence of the input on the output is minimized. The…
This work proposes A$^2$GD, a novel adaptive accelerated gradient descent method for convex and composite optimization. Smoothness and convexity constants are updated via Lyapunov analysis. Inspired by stability analysis in ODE solvers, the…
This paper is concerned with the distributed control and stabilization problems for linear discrete-time large scale systems with imposed constraints. The main contributions of this paper are: Firstly, by using the maximum principle…
The objective of this paper is to enhance the optimization process for neural networks by developing a dynamic learning rate algorithm that effectively integrates exponential decay and advanced anti-overfitting strategies. Our primary…
A framework is introduced for solving a sequence of slowly changing optimization problems, including those arising in regression and classification applications, using optimization algorithms such as stochastic gradient descent (SGD). The…
This paper provides a systematic exposition of Lyapunov stability for compact sets in locally compact metric spaces. We explore foundational concepts, including neighborhoods of compact sets, invariant sets, and the properties of dynamical…
In this paper, we present new results on finite- and fixed-time convergence for dynamical systems using LaSalle-like invariance principles. In particular, we provide first and second-order non-smooth Lyapunov-like results for finite- and…
Tilt stability is a fundamental concept of variational analysis and optimization that plays a pivotal role in both theoretical issues and numerical computations. This paper investigates tilt stability of local minimizers for a general class…
This paper considers time-average optimization, where a decision vector is chosen every time step within a (possibly non-convex) set, and the goal is to minimize a convex function of the time averages subject to convex constraints on these…
We provide Lyapunov-like characterizations of boundedness and convergence of non-trivial solutions for a class of systems with unstable invariant sets. Examples of systems to which the results may apply include interconnections of stable…
Although adaptive optimization algorithms have been successful in many applications, there are still some mysteries in terms of convergence analysis that have not been unraveled. This paper provides a novel non-convex analysis of adaptive…
The automatic selection of an appropriate time step size has been considered extensively in the literature. However, most of the strategies developed operate under the assumption that the computational cost (per time step) is independent of…
Classical adaptive control proves total-system stability for control of linear plants, but only for plants meeting very restrictive assumptions. Approximate Dynamic Programming (ADP) has the potential, in principle, to ensure stability…
Recently, there has been an increasing interest in using tools from dynamical systems to analyze the behavior of simple optimization algorithms such as gradient descent and accelerated variants. This paper strengthens such connections by…
In this paper we consider the problem of global asymptotic stabilization with prescribed local behavior. We show that this problem can be formulated in terms of control Lyapunov functions. Moreover, we show that if the local control law has…
Stochastic optimization has found wide applications in minimizing objective functions in machine learning, which motivates a lot of theoretical studies to understand its practical success. Most of existing studies focus on the convergence…
We study the problem of robust global stabilization in control-affine systems, focusing on dynamic uncertainties in the control directions \emph{and} the presence of topological obstructions that prevent the existence of smooth global…
Stochastic approximation (SA) and stochastic gradient descent (SGD) algorithms are work-horses for modern machine learning algorithms. Their constant stepsize variants are preferred in practice due to fast convergence behavior. However,…
The paper is concerned with the development of Lyapunov methods for the analysis of equilibrium stability in a dynamical system on the space of probability measures driven by a non-local continuity equation. We derive sufficient conditions…
In continual learning, a learner has to keep learning from the data over its whole life time. A key issue is to decide what knowledge to keep and what knowledge to let go. In a neural network, this can be implemented by using a step-size…