Related papers: Threshold Disorder as a Source of Diverse and Comp…
Chaotic pattern dynamics in many experimental systems show structured time averages. We suggest that simple universal boundary effects underly this phenomenon and exemplify them with the Kuramoto-Sivashinsky equation in a finite domain. As…
Training recurrent neural networks (RNNs) is a hard problem due to degeneracies in the optimization landscape, a problem also known as vanishing/exploding gradients. Short of designing new RNN architectures, previous methods for dealing…
We introduce the concept of Random Sequential Renormalization (RSR) for arbitrary networks. RSR is a graph renormalization procedure that locally aggregates nodes to produce a coarse grained network. It is analogous to the (quasi-)parallel…
We investigate the properties of an autoassociative network of threshold-linear units whose synaptic connectivity is spatially structured and asymmetric. Since the methods of equilibrium statistical mechanics cannot be applied to such a…
Recurrent Neural Networks (RNNs) are widely used for online regression due to their ability to generalize nonlinear temporal dependencies. As an RNN model, Long-Short-Term-Memory Networks (LSTMs) are commonly preferred in practice, as these…
Neuromorphic computing has recently gained momentum with the emergence of various neuromorphic processors. As the field advances, there is an increasing focus on developing training methods that can effectively leverage the unique…
Deep Recurrent Neural Network architectures, though remarkably capable at modeling sequences, lack an intuitive high-level spatio-temporal structure. That is while many problems in computer vision inherently have an underlying high-level…
Complex systems, when poised near a critical point of a phase transition between order and disorder, exhibit a dynamics comprising a scale-free mixture of order and disorder which is universal, i.e. system-independent (1-5). It allows…
Networks of strongly-coupled neurons with random connectivity exhibit chaotic, asynchronous fluctuations. In previous work, we showed that when endowed with an additional low-rank connectivity consisting of the outer product of orthogonal…
The emergence of large-scale spatial modulations of turbulent channel flow, as the Reynolds number is decreased, is addressed numerically using the framework of linear stability analysis. Such modulations are known as the precursors of…
Anomaly detection to recognize unusual events in large scale systems in a time sensitive manner is critical in many industries, eg. bank fraud, enterprise systems, medical alerts, etc. Large-scale systems often grow in size and complexity…
Thresholding--the pruning of nodes or edges based on their properties or weights--is an essential preprocessing tool for extracting interpretable structure from complex network data, yet existing methods face several key limitations.…
A simple threshold model of neuron firing (with the neuron membrane electrochemical potential governed by the chaotic Rossler attractor) has been analyzed by mapping the generated irregular spiking time-series into telegraph signals. In…
It has been argued by Thom and Palm that sparsely-connected neural networks (SCNs) show improved performance over fully-connected networks (FCNs). Super-regular networks (SRNs) are neural networks composed of a set of stacked sparse layers…
Weighted networks capture the structure of complex systems where interaction strength is meaningful. This information is essential to a large number of processes, such as threshold dynamics, where link weights reflect the amount of…
Physical systems experience nonlinear disturbances which have the potential to disrupt desired behavior. For a particular disturbance, whether or not the system recovers from the disturbance to a desired stable equilibrium point depends on…
Syntactic rules in natural language typically need to make reference to hierarchical sentence structure. However, the simple examples that language learners receive are often equally compatible with linear rules. Children consistently…
Despite the prevalence of the attention sink phenomenon in Large Language Models (LLMs), where initial tokens disproportionately monopolize attention scores, its structural origins remain elusive. This work provides a \textit{mechanistic…
We investigate the equilibria of a random model network exhibiting extensive chaos. In this regime, a large number of equilibria is present. They are all saddles with low-dimensional unstable manifolds. Surprisingly, despite network's…
The extraordinary computational power of the brain may be related in part to the fact that each of the smaller neural networks that compose it can behave transiently in many different ways, depending on its inputs. Mathematically, input…