Related papers: Neural Network-based High-index Saddle Dynamics Me…

Recent focus on robustness to adversarial attacks for deep neural networks produced a large variety of algorithms for training robust models. Most of the effective algorithms involve solving the min-max optimization problem for training…

Advancing the dynamics inference of power electronic systems (PES) to the real-time edge-side holds transform-ative potential for testing, control, and monitoring. How-ever, efficiently inferring the inherent hybrid continu-ous-discrete…

Here we present a multiscale method to calculate the saddle point associated with the effective dynamics arising from a stochastic system which couples slow deterministic drift and fast stochastic dynamics. This problem is motivated by the…

The Nested Dirichlet Distribution (NDD) provides a flexible alternative to the Dirichlet distribution for modeling compositional data, relaxing constraints on component variances and correlations through a hierarchical tree structure. While…

This paper aims to solve a distributed resource allocation problem with binary local constraints. The problem is formulated as a binary program with a cost function defined by the summation of agent costs plus a global mismatch/penalty…

Chemical reactions in multidimensional systems are often described by a rank-1 saddle, whose stable and unstable manifolds intersect in the normally hyperbolic invariant manifold (NHIM). Trajectories started on the NHIM in principle never…

Momentum based stochastic gradient methods such as heavy ball (HB) and Nesterov's accelerated gradient descent (NAG) method are widely used in practice for training deep networks and other supervised learning models, as they often provide…

In this work, we propose an efficient nullspace-preserving saddle search (NPSS) method for a class of phase transitions involving translational invariance, where the critical states are often degenerate. The NPSS method includes two stages,…

Complex networks describe important structures in nature and society, composed of nodes and the edges that connect them. The evolution of these networks is typically described by dynamics, which are labor-intensive and require expert…

We present a class of simple algorithms that allows to find the reaction path in systems with a complex potential energy landscape. The approach does not need any knowledge on the product state and does not require the calculation of any…

We prove error estimates for the semi-implicit numerical scheme of sphere-constrained high-index saddle dynamics, which serves as a powerful instrument in finding saddle points and constructing the solution landscapes of constrained systems…

We propose a derivative-free saddle-search algorithm designed to locate transition states using only function evaluations. The algorithm employs a nested architecture consisting of an inner eigenvector search and an outer saddle-point…

Nonconvex optimization problems such as the ones in training deep neural networks suffer from a phenomenon called saddle point proliferation. This means that there are a vast number of high error saddle points present in the loss function.…

Stochastic gradient descent (SGD) is a standard optimization method to minimize a training error with respect to network parameters in modern neural network learning. However, it typically suffers from proliferation of saddle points in the…

We develop and analyze numerical discretization to the constrained high-index saddle dynamics, the dynamics searching for the high-index saddle points confined on the high-dimensional unit sphere. Compared with the saddle dynamics without…

Non-convex optimal control arises from various applications but may contain multiple stationary points. Classical solvers usually perform a ``local'' search near a saddle point or a local minimum, thus rely on good initial guess to reach…

In light of the increasing adoption of edge computing in areas such as intelligent furniture, robotics, and smart homes, this paper introduces HyperSNN, an innovative method for control tasks that uses spiking neural networks (SNNs) in…

This paper develops a unified distributed method for solving two classes of constrained networked optimization problems, i.e., optimal consensus problem and resource allocation problem with non-identical set constraints. We first transform…

Policy gradient methods are an appealing approach in reinforcement learning because they directly optimize the cumulative reward and can straightforwardly be used with nonlinear function approximators such as neural networks. The two main…

Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biological interpretability. Their rich spatio-temporal information processing capability and event-driven nature make them ideally…