Related papers: Retrieval dynamics of neural networks for sparsely…
The synchronous dynamics and the stationary states of a recurrent attractor neural network model with competing synapses between symmetric sequence processing and Hebbian pattern reconstruction is studied in this work allowing for the…
Recurrent neural policies are widely used in partially observable control and meta-RL tasks. Their abilities to maintain internal memory and adapt quickly to unseen scenarios have offered them unparalleled performance when compared to…
Many social and other networks exhibit stable size scaling relationships, such that features such as mean degree or reciprocation rates change slowly or are approximately constant as the number of vertices increases. Statistical network…
Exactly solvable neural network models with asymmetric weights are rare, and exact solutions are available only in some mean-field approaches. In this article we find exact analytical solutions of an asymmetric spin-glass-like model of…
The classic paradigms for learning and memory recall focus on strengths of synaptic couplings and how these can be modulated to encode memories. In a previous paper [A. K. Behera, M. Rao, S. Sastry, and S. Vaikuntanathan, Physical Review X…
The task of a neural associative memory is to retrieve a set of previously memorized patterns from their noisy versions using a network of neurons. An ideal network should have the ability to 1) learn a set of patterns as they arrive, 2)…
For infinitely large sparse networks of spiking neurons mean field theory shows that a balanced state of highly irregular activity arises under various conditions. Here we analytically investigate the microscopic irregular dynamics in…
Reservoir computing is a popular approach to design recurrent neural networks, due to its training simplicity and approximation performance. The recurrent part of these networks is not trained (e.g., via gradient descent), making them…
We study with numerical simulation the possible limit behaviors of synchronous discrete-time deterministic recurrent neural networks composed of N binary neurons as a function of a network's level of dilution and asymmetry. The network…
We report on both analytical and numerical results concerning stochastic Hopfield--like neural automata exhibiting the following (biologically inspired) features: (1) Neurons and synapses evolve in time as in contact with respective baths…
Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits, yet classical results, such as Hopfield's model of associative memory, rely on symmetric connectivity that…
Recurrent neural networks show state-of-the-art results in many text analysis tasks but often require a lot of memory to store their weights. Recently proposed Sparse Variational Dropout eliminates the majority of the weights in a…
Advancements in artificial intelligence call for a deeper understanding of the fundamental mechanisms underlying deep learning. In this work, we propose a theoretical framework to analyze learning dynamics through the lens of dynamical…
The two-parameter dynamical replica theory (2-DRT) is applied to investigate retrieval properties of non-monotonic associative memory, a model which lacks thermodynamic potential functions. 2-DRT reproduces dynamical properties of the model…
We study the collective dynamics of a Leaky Integrate and Fire network in which precise relative phase relationship of spikes among neurons are stored, as attractors of the dynamics, and selectively replayed at differentctime scales. Using…
This paper is concerned with the modeling and analysis of two of the most commonly used recurrent neural network models (i.e., Hopfield neural network and firing-rate neural network) with dynamic recurrent connections undergoing Hebbian…
We investigate recurrent neural networks with asymmetric interactions and demonstrate that the inclusion of self-couplings or sparse excitatory inter-module connections leads to the emergence of a densely connected manifold of dynamically…
Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task related neural dynamics we study trained Recurrent Neural Networks. We develop a Mean Field Theory for Reservoir Computing…
Threshold rules of spreading in binary-state networks lead to cascades. We study persistent cascade-recovery dynamics on quasi-robust networks, i.e., networks which are robust against small trigger but may collapse for larger one. It is…
The deterministic dynamics of randomly connected neural networks are studied, where a state of binary neurons evolves according to a discreet-time synchronous update rule. We give a theoretical support that the overlap of systems' states…