Related papers: Locating saddle points using gradient extremals on…

Continuous time primal-dual gradient dynamics that find a saddle point of a Lagrangian of an optimization problem have been widely used in systems and control. While the global asymptotic stability of such dynamics has been well-studied, it…

Adaptive gradient methods have attracted much attention of machine learning communities due to the high efficiency. However their acceleration effect in practice, especially in neural network training, is hard to analyze, theoretically. The…

In this paper we consider solving saddle point problems using two variants of Gradient Descent-Ascent algorithms, Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA) methods. We show that both of these algorithms admit a…

We present a fast method for nonlinear data-driven model reduction of dynamical systems onto their slowest nonresonant spectral submanifolds (SSMs). We use observed data to locate a low-dimensional, attracting slow SSM and compute a…

Finding index-1 saddle points is crucial for understanding phase transitions. In this work, we propose a simple yet efficient approach, the spring pair method (SPM), to accurately locate saddle points. Without requiring Hessian information,…

In Hamiltonian systems subjected to periodic perturbations the stable and unstable manifolds of the unstable periodic orbits provide the dynamical "skeleton" that drives the mixing process and bounds the chaotic regions of the phase space.…

We derive a variational model to fit a composite B\'ezier curve to a set of data points on a Riemannian manifold. The resulting curve is obtained in such a way that its mean squared acceleration is minimal in addition to remaining close the…

Branch-specific substitution models are popular for detecting evolutionary change-points, such as shifts in selective pressure. However, applying such models typically requires prior knowledge of change-point locations on the phylogeny or…

Trajectory optimization methods have achieved an exceptional level of performance on real-world robots in recent years. These methods heavily rely on accurate analytical models of the dynamics, yet some aspects of the physical world can…

This paper describes a novel approach to change-point detection when the observed high-dimensional data may have missing elements. The performance of classical methods for change-point detection typically scales poorly with the…

One-class learning is the classic problem of fitting a model to data for which annotations are available only for a single class. In this paper, we propose a novel objective for one-class learning. Our key idea is to use a pair of…

In this paper, we propose a point cloud classification method based on graph neural network and manifold learning. Different from the conventional point cloud analysis methods, this paper uses manifold learning algorithms to embed point…

Lens designers routinely use optimization in their everyday practice. Local optimization algorithms lead to the nearest minimum. In this paper, a new deterministic approach for multi-extremum optimization is proposed. Optimal solutions for…

In this work, we investigate the use of data-driven equation discovery for dynamical systems to model and forecast continuous-time dynamics of unconstrained optimization problems. To avoid expensive evaluations of the objective function and…

We consider distributed smooth nonconvex unconstrained optimization over networks, modeled as a connected graph. We examine the behavior of distributed gradient-based algorithms near strict saddle points. Specifically, we establish that (i)…

There are several applications of stochastic optimization where one can benefit from a robust estimate of the gradient. For example, domains such as distributed learning with corrupted nodes, the presence of large outliers in the training…

Recent work has shown that stochastically perturbed gradient methods can efficiently escape strict saddle points of smooth functions. We extend this body of work to nonsmooth optimization, by analyzing an inexact analogue of a…

Saddle points play important roles as the transition states of activated process in gradient system driven by energy functional. However, for the same energy functional, the saddle points, as well as other stationary points, are different…

We derive the system of differential equations for the gradient flow characterizing the training process of linear in-context learning in full generality. Next, we explore the geometric structure of the gradient flows in two instances,…

Nonconvex optimization underlies many modern machine learning and control tasks, where saddle points pose the dominant obstacle to reliable convergence in high-dimensional settings. Escaping these saddle points deterministically using…