Related papers: The connections between Lyapunov functions for som…
We study the problem of minimizing a sum of local objective convex functions over a network of processors/agents. This problem naturally calls for distributed optimization algorithms, in which the agents cooperatively solve the problem…
We use differential equations based approaches to provide some {\it \textbf{physics}} insights into analyzing the dynamics of popular optimization algorithms in machine learning. In particular, we study gradient descent, proximal gradient…
In this paper we propose a new class of iterative regularization methods for solving ill-posed linear operator equations. The prototype of these iterative regularization methods is in the form of second order evolution equation with a…
This work aims to minimize a continuously differentiable convex function with Lipschitz continuous gradient under linear equality constraints. The proposed inertial algorithm results from the discretization of the second-order primal-dual…
The rapid development of large language models (LLMs) has driven the demand for more efficient optimization techniques. Among these, the Lookahead family of optimizers employs a two-loop framework, maintaining fast and slow sets of model…
We apply mixed-precision to the low-rank Lyapunov ADI (LR-ADI) by performing certain aspects of the algorithm in a lower working precision. Namely, we accumulate the overall solution, solve the linear systems comprising the ADI iteration,…
This paper considers a class of structured fractional minimization problems. The numerator consists of a differentiable function, a simple nonconvex nonsmooth function, a concave nonsmooth function, and a convex nonsmooth function composed…
A class of second-order algorithms is proposed for minimizing smooth nonconvex functions that alternates between regularized Newton and negative curvature steps in an iteration-dependent subspace. In most cases, the Hessian matrix is…
Three similar convergence notions are considered. Two of them are the long established notions of convergent dynamics and incremental stability. The other is the more recent notion of contraction analysis. All three convergence notions…
In this paper, we focus on providing convergence guarantees for stochastic subgradient methods in minimizing nonsmooth nonconvex functions. We first investigate the global stability of a general framework for stochastic subgradient methods,…
We investigate the asymptotic properties of the trajectories generated by a second-order dynamical system with Hessian driven damping and a Tikhonov regularization term in connection with the minimization of a smooth convex function in…
In the dynamical systems approach to describing turbulent or otherwise chaotic flows, an important quantity is the Lyapunov exponents and vectors that characterize the strange attractor of the flow. In particular, knowledge of the Lyapunov…
We introduce and analyze an algorithm for the minimization of convex functions that are the sum of differentiable terms and proximable terms composed with linear operators. The method builds upon the recently developed smoothed gap…
We propose a method for training ordinary differential equations by using a control-theoretic Lyapunov condition for stability. Our approach, called LyaNet, is based on a novel Lyapunov loss formulation that encourages the inference…
Viewing optimization methods as numerical integrators for ordinary differential equations (ODEs) provides a thought-provoking modern framework for studying accelerated first-order optimizers. In this literature, acceleration is often…
In this paper, in the setting of Hilbert spaces, we consider a Tikhonov regularized second-order plus first-order primal-dual dynamical system with asymptotically vanishing damping for a linear equality constrained convex optimization…
We study the convergence analysis for general degenerate and non-reversible stochastic differential equations (SDEs). We apply the Lyapunov method to analyze the Fokker-Planck equation, in which the Lyapunov functional is chosen as a…
The state-of-the art proof of a global inf-sup condition on mixed finite element schemes does not allow for an analysis of truly indefinite, second-order linear elliptic PDEs. This paper, therefore, first analyses a nonconforming finite…
We extend the Lyapunov stability criterion to Euler discretizations of differential inclusions. It relies on a pair of Lyapunov functions, one in continuous time and one in discrete time. In the context of optimization, this yields…
We extend the standard notion of self-concordance to non-convex optimization and develop a family of second-order algorithms with global convergence guarantees. In particular, two function classes -- \textit{weakly self-concordant}…