Related papers: Emergent Higher-Order Structure from Fast Adaptive…

The Primal-Dual (PD) algorithm is widely used in convex optimization to determine saddle points. While the stability of the PD algorithm can be easily guaranteed, strict contraction is nontrivial to establish in most cases. This work…

In this paper, we provide some analysis on the asymptotic escape of strict saddles in manifold optimization using the projected gradient descent (PGD) algorithm. One of our main contributions is that we extend the current analysis to…

Adaptive (or co-evolutionary) network dynamics, i.e., when changes of the network/graph topology are coupled with changes in the node/vertex dynamics, can give rise to rich and complex dynamical behavior. Even though adaptivity can improve…

Dissipative partial differential equations that exhibit chaotic dynamics tend to evolve to attractors that exist on finite-dimensional manifolds. We present a data-driven reduced order modeling method that capitalizes on this fact by…

We construct a nontrivial generalization of the paradigmatic Kuramoto model by using an additional coupling term that explicitly breaks its rotational symmetry resulting in a variant of the Winfree Model. Consequently, we observe the…

The Ott-Antonsen ansatz shows that, for certain classes of distribution of the natural frequencies in systems of $N$ globally coupled Kuramoto oscillators, the dynamics of the order parameter, in the limit $N\to \infty$, evolves, under…

Federated learning has attracted increasing attention with the emergence of distributed data. While extensive federated learning algorithms have been proposed for the non-convex distributed problem, federated learning in practice still…

Despite recent advances, the remaining bottlenecks in deep generative models are necessity of extensive training and difficulties with generalization from small number of training examples. We develop a new generative model called…

Streaming adaptations of manifold learning based dimensionality reduction methods, such as Isomap, are based on the assumption that a small initial batch of observations is enough for exact learning of the manifold, while remaining…

The slow drift along a manifold of periodic orbits is a key mathematical structure underlying bursting dynamics in many scientific applications. While classical averaging theory, as formalised by the Pontryagin-Rodygin theorem, provides a…

The increasing size of neural networks has led to a growing demand for methods of efficient fine-tuning. Recently, an orthogonal fine-tuning paradigm was introduced that uses orthogonal matrices for adapting the weights of a pretrained…

This work develops the global equations of neural networks through stacked piecewise manifolds, fixed-point theory, and boundary-conditioned iteration. Once fixed coordinates and operators are removed, a neural network appears as a…

Canards, special trajectories that follow invariant repelling slow manifolds for long time intervals, have been frequently observed in slow-fast systems of either biological, chemical and physical nature. Here, collective canard explosions…

Recent work on mode connectivity in the loss landscape of deep neural networks has demonstrated that the locus of (sub-)optimal weight vectors lies on continuous paths. In this work, we train a neural network that serves as a hypernetwork,…

By incorporating physical consistency as inductive bias, deep neural networks display increased generalization capabilities and data efficiency in learning nonlinear dynamic models. However, the complexity of these models generally…

Adaptive gradient methods have attracted much attention of machine learning communities due to the high efficiency. However their acceleration effect in practice, especially in neural network training, is hard to analyze, theoretically. The…

The problem of identifying geometric structure in data is a cornerstone of (unsupervised) learning. As a result, Geometric Representation Learning has been widely applied across scientific and engineering domains. In this work, we…

Power grids, as well as neuronal networks with synaptic plasticity, describe real-world systems of tremendous importance for our daily life. The investigation of these seemingly unrelated types of dynamical networks has attracted increasing…

In the past two decades, a series of important results have been established in the empirical and theoretical modeling of complex networks, although considered are mainly pairwise networks. However, with the development of science and…

Network topology plays a key role in many phenomena, from the spreading of diseases to that of financial crises. Whenever the whole structure of a network is unknown, one must resort to reconstruction methods that identify the least biased…