Related papers: Associative Memory by Recurrent Neural Networks wi…
We investigated how the stability of macroscopic states in the associative memory model is affected by synaptic depression. To this model, we applied the dynamical mean-field theory, which has recently been developed in stochastic neural…
Humans learn and form memories in stochastic environments. Auto-associative memory systems model these processes by storing patterns and later recovering them from corrupted versions. Here, memories are learned by associating each pattern…
We complement our previous work [arxiv: 0707.0565] with the full (non diluted) solution describing the stable states of an attractor network that stores correlated patterns of activity. The new solution provides a good fit of simulations of…
Learning how complex dynamical systems evolve over time is a key challenge in system identification. For safety critical systems, it is often crucial that the learned model is guaranteed to converge to some equilibrium point. To this end,…
The extension of deep learning towards temporal data processing is gaining an increasing research interest. In this paper we investigate the properties of state dynamics developed in successive levels of deep recurrent neural networks…
Time delays increase the effective dimensionality of reservoirs, thus suggesting that time delays in reservoirs can enhance their performance, particularly their memory and prediction abilities. We find new closed-form expressions for…
Neural dynamical systems with stable attractor structures, such as point attractors and continuous attractors, are hypothesized to underlie meaningful temporal behavior that requires working memory. However, working memory may not support…
Quantum memory is a central component for quantum information processing devices, and will be required to provide high-fidelity storage of arbitrary states, long storage times and small access latencies. Despite growing interest in applying…
We describe a method to model nonlinear dynamical systems using periodic solutions of delay-differential equations. We show that any finite-time trajectory of a nonlinear dynamical system can be loaded approximately into the initial…
The ability of discrete-time nonlinear recurrent neural networks to store time-varying small input signals is investigated by mean-field theory. The combination of a small input strength and mean-field assumptions makes it possible to…
We consider a simple Markovian class of the stochastic Wilson-Cowan type models of neuronal network dynamics, which incorporates stochastic delay caused by the existence of a refractory period of neurons. From the point of view of the…
When the scale of communication networks has been growing rapidly in the past decades, it becomes a critical challenge to extract fast and accurate estimation of key state parameters of network links, e.g., transmission delays and dropped…
Recent experimental studies indicate that synaptic changes induced by neuronal activity are discrete jumps between a small number of stable states. Learning in systems with discrete synapses is known to be a computationally hard problem.…
The comprehension of the mechanisms at the basis of the functioning of complexly interconnected networks represents one of the main goals of neuroscience. In this work, we investigate how the structure of recurrent connectivity influences…
Memories are stored, retained, and recollected through complex, coupled processes operating on multiple timescales. To understand the computational principles behind these intricate networks of interactions we construct a broad class of…
Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This…
Frame stacking is broadly applied in end-to-end neural network training like connectionist temporal classification (CTC), and it leads to more accurate models and faster decoding. However, it is not well-suited to conventional neural…
Attractor networks are an influential theory for memory storage in brain systems. This theory has recently been challenged by the observation of strong temporal variability in neuronal recordings during memory tasks. In this work, we study…
This paper presents the design of an associative memory with feedback that is capable of on-line temporal sequence learning. A framework for on-line sequence learning has been proposed, and different sequence learning models have been…
Memory is a complex phenomenon that involves several distinct mechanisms. These mechanisms operate at different spatial and temporal levels. This chapter focuses on the theoretical framework and the mathematical models that have been…