Related papers: Gradient flow approach to geometric convergence an…
We propose an optimization proxy in terms of iterative implicit gradient methods for solving constrained optimization problems with nonconvex loss functions. This framework can be applied to a broad range of machine learning settings,…
This article presents a novel resolution to the problem of spline interpolation versus least-squares fitting on smooth Riemannian manifolds utilizing the method of gradient flows of networks. This approach represents a contribution to both…
Bilevel optimization is a fundamental tool in hierarchical decision-making and has been widely applied to machine learning tasks such as hyperparameter tuning, meta-learning, and continual learning. While significant progress has been made…
The convergence of the so-called quadratic method for computing eigenvalue enclosures of general self-adjoint operators is examined. Explicit asymptotic bounds for convergence to isolated eigenvalues are found. These bounds turn out to…
Riemannian accelerated gradient methods have been well studied for smooth optimization, typically treating geodesically convex and geodesically strongly convex cases separately. However, their extension to nonsmooth problems on manifolds…
The Conjugate Gradient method (CGM) is known to be the fastest generic iterative method for solving linear systems with symmetric sign definite matrices. In this paper, we modify this method so that it could find fundamental solitary waves…
This paper addresses the problems of spline interpolation on smooth Riemannian manifolds, with or without the inclusion of least-squares fitting. Our unified approach utilizes gradient flows for successively connected curves or networks,…
Bilevel optimization problems are receiving increasing attention in machine learning as they provide a natural framework for hyperparameter optimization and meta-learning. A key step to tackle these problems is the efficient computation of…
A new iteration bound for the preconditioned conjugate gradient (PCG) method is presented that more accurately captures convergence for systems with clustered eigenspectra, where the classical condition number-based bound is too…
Recent works by Bot-Fadili-Nguyen (arXiv:2510.22715) and by Jang-Ryu (arXiv:2510.23513) resolve long-standing iterate convergence questions for accelerated (proximal) gradient methods. In particular, Bot-Fadili-Nguyen prove weak convergence…
We present variational approximations of boundary value problems for curvature flow (curve shortening flow) and elastic flow (curve straightening flow) in two-dimensional Riemannian manifolds that are conformally flat. For the evolving open…
We develop an iterative refinement method that improves the accuracy of a user-chosen subset of $k$ eigenvectors ($k\ll n$) of an $n\times n$ real symmetric matrix. Using an orthogonal matrix represented in compact WY form, the method…
For compact self-adjoint operators in Hilbert spaces, two algorithms are proposed to provide fully computable a posteriori error estimate for eigenfunction approximation. Both algorithms apply well to the case of tight clusters and multiple…
The constrained gradient method (CGM) has recently been proposed to solve convex optimization and monotone variational inequality (VI) problems with general functional constraints. While existing literature has established convergence…
We consider the optimization problem with a generally quadratic matrix constraint of the form $X^TAX = J$, where $A$ is a given nonsingular, symmetric $n\times n$ matrix and $J$ is a given $k\times k$ symmetric matrix, with $k\leq n$,…
The gradient method for minimize a differentiable convex function on Riemannian manifolds with lower bounded sectional curvature is analyzed in this paper. The analysis of the method is presented with three different finite procedures for…
We consider the numerical approximation of the spectrum of a second-order elliptic eigenvalue problem by the hybridizable discontinuous Galerkin (HDG) method. We show for problems with smooth eigenfunctions that the approximate eigenvalues…
Gradient-based iterative optimization methods are the workhorse of modern machine learning. They crucially rely on careful tuning of parameters like learning rate and momentum. However, one typically sets them using heuristic approaches…
We develop an accelerated gradient descent algorithm on the Grassmann manifold to compute the subspace spanned by a number of leading eigenvectors of a symmetric positive semi-definite matrix. This has a constant cost per iteration and a…
We perform a convergence analysis of a discrete-in-time minimization scheme approximating a finite dimensional singularly perturbed gradient flow. We allow for different scalings between the viscosity parameter $\varepsilon$ and the time…