Related papers: Improved high-index saddle dynamics for finding sa…
The decentralized gradient descent (DGD) algorithm, and its sibling, diffusion, are workhorses in decentralized machine learning, distributed inference and estimation, and multi-agent coordination. We propose a novel, principled framework…
Sampling the stationary points of a complicated potential energy landscape is a challenging problem. Here we introduce a sampling method based on relaxation from stationary points of the highest index of the Hessian matrix. We illustrate…
We study the asymptotic behavior of second-order algorithms mixing Newton's method and inertial gradient descent in non-convex landscapes. We show that, despite the Newtonian behavior of these methods, they almost always escape strict…
The classical theory of attractive central force problem on the standard (flat) Euclidean plane can be generalized to surfaces by reformulating the basic underlying physical principles by means of differential geometry. Attractive central…
Euclidean representation learning methods have achieved promising results in image fusion tasks, which can be attributed to their clear advantages in handling with linear space. However, data collected from a realistic scene usually has a…
We propose a hybrid physics-informed machine learning framework to approximate invariant manifolds (IMs) of discrete-time dynamical systems driven by exogenous autonomous dynamics (exosystems). Such systems appear in applications ranging…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…
Optimization algorithms are unlikely to converge to strict saddle points. Proofs to that effect rely on the Center-Stable Manifold Theorem (CSMT), casting algorithms as dynamical systems: $x_{k+1} = g_k(x_k)$. In its standard form, the CSMT…
Model reduction is a key technology for large-scale physical systems in science and engineering, as it brings behavior expressed in many degrees of freedom to a more manageable size that subsequently allows control, optimization, and…
How can we understand gradient-based training over non-convex landscapes? The edge of stability phenomenon, introduced in Cohen et al. (2021), indicates that the answer is not so simple: namely, gradient descent (GD) with large step sizes…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…
We develop new variational principles to study stability and equilibrium of axisymmetric flows. We show that there is an infinite number of steady state solutions. We show that these steady states maximize a (non-universal) $H$-function. We…
We present a new microscopic ODE-based model for pedestrian dynamics: the Gradient Navigation Model. The model uses a superposition of gradients of distance functions to directly change the direction of the velocity vector. The velocity is…
Accelerated gradient descent iterations are widely used in optimization. It is known that, in the continuous-time limit, these iterations converge to a second-order differential equation which we refer to as the accelerated gradient flow.…
This paper introduces a novel Homogeneous Second-order Descent Ascent (HSDA) algorithm for nonconvex-strongly concave minimax optimization problems. At each iteration, HSDA uniquely computes a search direction by solving a homogenized…
The Nielsen-Thurston theory of surface diffeomorphisms shows that useful dynamical information can be obtained about a surface diffeomorphism from a finite collection of periodic orbits.In this paper, we extend these results to homoclinic…
Many engineering problems involve learning hidden dynamics from indirect observations, where the physical processes are described by systems of partial differential equations (PDE). Gradient-based optimization methods are considered…
This paper introduces a preconditioned method designed to comprehensively address the saddle point system with the aim of improving convergence efficiency. In the preprocessor construction phase, a technical approach for solving the…
When a real saddle equilibrium in a three-dimensional vector field undergoes a homoclinic bifurcation, the associated two-dimensional invariant manifold of the equilibrium closes on itself in an orientable or non-orientable way. We are…
The Morse-Smale complex is a standard tool in visual data analysis. The classic definition is based on a continuous view of the gradient of a scalar function where its zeros are the critical points. These points are connected via gradient…