Related papers: Optimal sequence memory in driven random networks
Associative memory models retrieve stored information through content-based addressing, mimicking the neural processes of animal brains. The classical Hopfield network-based models store memories as vectors of discrete values and have good…
We investigate a phase transition from linear to nonlinear information processing in echo state networks, a widely used framework in reservoir computing. The network consists of randomly connected recurrent nodes perturbed by a noise and…
In this work we study, analytically and employing Monte Carlo simulations, the influence of the competition between several activity-dependent synaptic processes, such as short-term synaptic facilitation and depression, on the maximum…
Recurrent neural networks (RNNs) are non-linear dynamic systems. Previous work believes that RNN may suffer from the phenomenon of chaos, where the system is sensitive to initial states and unpredictable in the long run. In this paper,…
This study shows that a mixture of RNN experts model can acquire the ability to generate sequences combining multiple primitive patterns by means of self-organizing chaos. By training of the model, each expert learns a primitive sequence…
Reservoir computing systems, a class of recurrent neural networks, have recently been exploited for model-free, data-based prediction of the state evolution of a variety of chaotic dynamical systems. The prediction horizon demonstrated has…
Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task related neural dynamics we study trained Recurrent Neural Networks. We develop a Mean Field Theory for Reservoir Computing…
Understanding how deep neural networks learn remains a fundamental challenge in modern machine learning. A growing body of evidence suggests that training dynamics undergo a distinct phase transition, yet our understanding of this…
We investigate chaos in mixed-phase-space Hamiltonian systems using time series of the finite- time Lyapunov exponents. The methodology we propose uses the number of Lyapunov exponents close to zero to define regimes of ordered…
In this work and the supporting Parts II [2] and III [3], we provide a rather detailed analysis of the stability and performance of asynchronous strategies for solving distributed optimization and adaptation problems over networks. We…
The study of deterministic chaos continues to be one of the important problems in the field of nonlinear dynamics. Interest in the study of chaos exists both in low-dimensional dynamical systems and in large ensembles of coupled…
Although real-world complex systems typically interact through sparse and heterogeneous networks, analytic solutions of their dynamics are limited to models with all-to-all interactions. Here, we solve the dynamics of a broad range of…
We investigate the equilibria of a random model network exhibiting extensive chaos. In this regime, a large number of equilibria is present. They are all saddles with low-dimensional unstable manifolds. Surprisingly, despite network's…
This paper discusses mixing of chaotic systems as a dependable method for secure communication. Distribution of the entropy function for steady state as well as plaintext input sequences are analyzed. It is shown that the mixing of chaotic…
Until now, most memristor-based chaotic circuits proposed in the literature are based on mathematical models which assume ideal characteristics such as piece-wise linear or cubic non-linearities. The idea, illustrated here and originating…
Consistency and predictability of brain functionalities depend on reproducible activity of a single neuron. We identify a reproducible non-chaotic neuronal phase where deviations between concave response latency profiles of a single neuron…
Deciphering the underpinnings of the dynamical processes leading to information transmission, processing, and storing in the brain is a crucial challenge in neuroscience. An inspiring but speculative theoretical idea is that such dynamics…
Neural networks with recurrent asymmetric couplings are important to understand how episodic memories are encoded in the brain. Here, we integrate the experimental observation of wide synaptic integration window into our model of sequence…
We study randomly initialized residual networks using mean field theory and the theory of difference equations. Classical feedforward neural networks, such as those with tanh activations, exhibit exponential behavior on the average when…
Further analysis and experimentation is carried out in this paper for a chaotic dynamic model, viz. the Nonlinear Dynamic State neuron (NDS). The analysis and experimentations are performed to further understand the underlying dynamics of…