Related papers: Transient dynamics of associative memory models

We study the synchronous dynamics of the Hopfield model when a random antisymmetric part is added to the otherwise symmetric synaptic matrix. We use a generating functional technique to derive analytical expressions for the order parameters…

Time-dependent data-generating distributions have proven to be difficult for gradient-based training of neural networks, as the greedy updates result in catastrophic forgetting of previously learned knowledge. Despite the progress in the…

Using the Hopfield model as a benchmark case, the present work focuses on the investigation of partially annealed associative neural networks, wherein neural dynamics is coupled to slowly evolving patterns within the…

An exact solution of the transient dynamics for a sequential associative memory model is discussed through both the path-integral method and the statistical neurodynamics. Although the path-integral method has the ability to give an exact…

We propose a dynamical neural network model with a hierarchical and modular structure. The network architecture can be derived by minimizing an energy function that is originally designed based on two kinds of neurons with quite different…

Associative memory refers to the ability to relate a memory with an input and targets the restoration of corrupted patterns. It has been intensively studied in classical physical systems, as in neural networks where an attractor dynamics…

In neural circuits, statistical connectivity rules strongly depend on neuronal type. Here we study dynamics of neural networks with cell-type specific connectivity by extending the dynamic mean field method, and find that these networks…

Ever since the last two decades of the past century pioneering studies in the field of statistical physics had focused their efforts on developing models of neural networks that could display memory storage and retrieval. Though many…

We study associative memory neural networks of the Hodgkin-Huxley type of spiking neurons in which multiple periodic spatio-temporal patterns of spike timing are memorized as limit-cycle-type attractors. In encoding the spatio-temporal…

Whereas the importance of transient dynamics to the functionality and management of complex systems has been increasingly recognized, most of the studies are based on models. Yet in realistic situations the models are often unknown and what…

We study associative memory based on temporal coding in which successful retrieval is realized as an entrainment in a network of simple phase oscillators with distributed natural frequencies under the influence of white noise. The memory…

We consider nonequilibrium transport in a simple chain of identical mechanical cells in which particles move around. In each cell, there is a rotating disc, with which these particles interact, and this is the only interaction in the model.…

Gradient descent dynamics in complex energy landscapes, i.e. featuring multiple minima, finds application in many different problems, from soft matter to machine learning. Here, we analyze one of the simplest examples, namely that of soft…

Generative models, including diffusion models, are increasingly used as foundation models and adapted through sequential fine-tuning, making continual learning an essential problem setting. However, continual learning in such generative…

We analyze the input-output behavior of residual networks from a dynamical system point of view by disentangling the residual dynamics from the output activities before the classification stage. For a network with simple skip connections…

It is shown that a Hopfield recurrent neural network exhibits a scaling regime, whose specific exponents depend on the number of parcels used and the decay length of the coupling strength. This scaling regime recovers the picture introduced…

Neurons in the brain communicate with spikes, which are discrete events in time and value. Functional network models often employ rate units that are continuously coupled by analog signals. Is there a qualitative difference implied by these…

Several approaches to cognition and intelligence research rely on statistics-based models testing, namely factor analysis. In the present work we exploit the emerging dynamical systems perspective putting the focus on the role of the…

Recent advances at the intersection of control theory, neuroscience, and machine learning have revealed novel mechanisms by which dynamical systems perform computation. These advances encompass a wide range of conceptual, mathematical, and…

Transformers are one of the most successful architectures of modern neural networks. At their core there is the so-called attention mechanism, which recently interested the physics community as it can be written as the derivative of an…