Related papers: Transient amplification in balanced neural network…
We investigate dynamical systems characterized by a time series of distinct semi-stable activity patterns, as they are observed in cortical neural activity patterns. We propose and discuss a general mechanism allowing for an adiabatic…
Understanding how local perturbations induce the transient dynamics of a network of coupled units is essential to control and operate such systems. Often a perturbation initiated in one unit spreads to other units whose dynamical state they…
Transient growth mechanisms operating on streaky shear flows are believed important for sustaining near-wall turbulence. Of the three individual mechanisms present - Orr, lift-up and 'push over' - Lozano-Duran et. al. (J. Fluid Mech. 914,…
We study signal propagation at initialization in transformers through the averaged partial Jacobian norm (APJN), a measure of gradient amplification across layers. We extend APJN analysis to transformers with bidirectional attention and…
We study physical mechanisms that trigger transient growth in a high-speed spatially-developing laminar boundary layer that interacts with an oblique shock wave. We utilize an approach based on power-iteration, with the global forward and…
Even if a linear system of ordinary differential equations has a globally attracting equilibrium at the origin, small disturbances from the equilibrium may lead to large transient excursions before the system stabilizes. This…
We investigate choice-driven network growth. In this model, nodes are added one by one according to the following procedure: for each addition event a set of target nodes is selected, each according to linear preferential attachment, and a…
Transformer-based models have demonstrated exceptional performance across diverse domains, becoming the state-of-the-art solution for addressing sequential machine learning problems. Even though we have a general understanding of the…
In search of many social and economical systems, it is found that node strength distribution as well as degree distribution demonstrate the behavior of power-law with droop-head and heavy-tail. We present a new model for the growth of…
In the present study, an amplifying neuron and attenuating neuron, which can be easily implemented into neural networks without any significant additional computational effort, are proposed. The activated output value is squared for the…
Random walks constitute a fundamental mechanism for a large set of dynamics taking place on networks. In this article, we study random walks on weighted networks with an arbitrary degree distribution, where the weight of an edge between two…
The thermal ratchets model toggles a spatially periodic asymmetric potential to rectify random walks and achieve transport of diffusing particles. We numerically solve the governing equation for the full dynamics of an infinite 1D ratchet…
We develop an approach to efficiently grow neural networks, within which parameterization and optimization strategies are designed by considering their effects on the training dynamics. Unlike existing growing methods, which follow simple…
Differential equations are a ubiquitous tool to study dynamics, ranging from physical systems to complex systems, where a large number of agents interact through a graph with non-trivial topological features. Data-driven approximations of…
Large and diverse datasets have been the cornerstones of many impressive advancements in artificial intelligence. Intelligent creatures, however, learn by interacting with the environment, which changes the input sensory signals and the…
A new approach for efficiently exploring the configuration space and computing the free energy of large atomic and molecular systems is proposed, motivated by an analogy with reinforcement learning. There are two major components in this…
We investigate the localization observed recently for locally non-hermitian Hamiltonians by studying the effect of the amplification on the scaling behavior of the transmission and reflection phases in 1D periodic chains of…
We analyse a system composed of a qubit coupled to electromagnetic fields of two high quality quantum oscillators. Particular realization of such a system is the superconducting qubit coupled to a transmission-line resonator driven by two…
Various functions of a network of excitable units can be enhanced if the network is in the `critical regime', where excitations are, on average, neither damped nor amplified. An important question is how can such networks self-organize to…
This work studies how to preemptively increase the resilience of a network by means of time-varying topological actuation. To do this, we focus on linear dynamical systems that are compatible with a given network, and consider policies that…