Related papers: Optimal sequence memory in driven random networks
In the human brain, internal states are often correlated over time (due to local recurrence and other intrinsic circuit properties), punctuated by abrupt transitions. At first glance, temporal smoothness of internal states presents a…
The behavior of the network and its stability are governed by both dynamics of individual nodes as well as their topological interconnections. Attention mechanism as an integral part of neural network models was initially designed for…
Chaos is generic in strongly-coupled recurrent networks of model neurons, and thought to be an easily accessible dynamical regime in the brain. While neural chaos is typically seen as an impediment to robust computation, we show how such…
We observe a novel 'multiple-descent' phenomenon during the training process of LSTM, in which the test loss goes through long cycles of up and down trend multiple times after the model is overtrained. By carrying out asymptotic stability…
We predict that continuously monitored quantum dynamics can be chaotic. The optimal paths between past and future boundary conditions can diverge exponentially in time when there is time-dependent evolution and continuous weak monitoring.…
Dynamical systems driven by strong external signals are ubiquituous in nature and engineering. Here we study "echo state networks", networks of a large number of randomly connected nodes, which represent a simple model of a neural network,…
Chaotic dynamics is always characterized by swarms of unstable trajectories, unpredictable individually, and thus generally studied statistically. It is often the case that such phase-space densities relax exponentially fast to a limiting…
Memories in neural system are shaped through the interplay of neural and learning dynamics under external inputs. By introducing a simple local learning rule to a neural network, we found that the memory capacity is drastically increased by…
The goal of polyhomeostatic control is to achieve a certain target distribution of behaviors, in contrast to polyhomeostatic regulation which aims at stabilizing a steady-state dynamical state. We consider polyhomeostasis for individual and…
In randomly connected networks of pulse-coupled elements a time-dependent input signal can be buffered over a short time. We studied the signal buffering properties in simulated networks as a function of the networks state, characterized by…
Classical chaos theory rests on the notion of universality, whereby disparate dynamical systems share identical scaling laws. Existing universality classes, however, implicitly assume Markovian dynamics. Here, a logistic map endowed with…
What is the reason for complex dynamical patterns registered from real biological neuronal networks? Noise and dynamical reconfiguring of a network (functional/dynamic connectome) were proposed as possible answers. In this case study, we…
We investigate the role of connection density in an adaptive network model of chaotic units that dynamically rewire based on their internal states and local coherence. By systematically varying the network's connectivity density, we uncover…
Chaos is an inherently dynamical phenomenon traditionally studied for trajectories that are either permanently erratic or transiently influenced by permanently erratic ones lying on a set of measure zero. The latter gives rise to the final…
Competitive neural networks are often used to model the dynamics of perceptual bistability. Switching between percepts can occur through fluctuations and/or a slow adaptive process. Here, we analyze switching statistics in competitive…
A massively recurrent neural network responds on one side to input stimuli and is autonomously active, on the other side, in the absence of sensory inputs. Stimuli and information processing depends crucially on the qualia of the…
As a phenomenon in dynamical systems allowing autonomous switching between stable behaviors, chaotic itinerancy has gained interest in neurorobotics research. In this study, we draw a connection between this phenomenon and the predictive…
Networks of chaotic units with static couplings can synchronize to a common chaotic trajectory. The effect of dynamic adaptive couplings on the cooperative behavior of chaotic networks is investigated. The couplings adjust to the activities…
Recurrent Neural Networks (RNNs) frequently exhibit complicated dynamics, and their sensitivity to the initialization process often renders them notoriously hard to train. Recent works have shed light on such phenomena analyzing when…
The practical success of widely used machine learning (ML) and deep learning (DL) algorithms in Artificial Intelligence (AI) community owes to availability of large datasets for training and huge computational resources. Despite the…