Related papers: Transient dynamics of associative memory models
Dense associative memory, a fundamental instance of modern Hopfield networks, can store a large number of memory patterns as equilibrium states of recurrent networks. While the stationary-state storage capacity has been investigated, its…
High-order extensions of the Hopfield model are known to exhibit dramatically enhanced storage capacity at equilibrium, while their dynamical retrieval properties remain less well understood. In our previous work, we carried out a dynamical…
Associative memory, a form of content-addressable memory, facilitates information storage and retrieval in many biological and physical systems. In statistical mechanics models, associative memory at equilibrium is represented through…
Transformer-based models have demonstrated exceptional performance across diverse domains, becoming the state-of-the-art solution for addressing sequential machine learning problems. Even though we have a general understanding of the…
The Hopfield model provides a mathematically idealized yet insightful framework for understanding the mechanisms of memory storage and retrieval in the human brain. This model has inspired four decades of extensive research on learning and…
Dense Associative Memories or Modern Hopfield Networks have many appealing properties of associative memory. They can do pattern completion, store a large number of memories, and can be described using a recurrent neural network with a…
The Hopfield associative memory model stores random patterns in synaptic couplings according to Hebb's rule and retrieves them through gradient descent on an energy function. This conventional setting, where neurons are assumed to have…
We study a fully connected Hopfield-type associative memory network with online activity-dependent synaptic plasticity, where neural states and synaptic couplings coevolve during retrieval. Using the generating-functional formalism, we…
Understanding the memory capacity of neural networks remains a challenging problem in implementing artificial intelligence systems. In this paper, we address the notion of capacity with respect to Hopfield networks and propose a dynamic…
We present a Hopfield-like autoassociative network for memories representing examples of concepts. Each memory is encoded by two activity patterns with complementary properties. The first is dense and correlated across examples within…
Recent generalizations of the Hopfield model of associative memories are able to store a number $P$ of random patterns that grows exponentially with the number $N$ of neurons, $P=\exp(\alpha N)$. Besides the huge storage capacity, another…
The Hopfield model is a pioneering neural network model with associative memory retrieval. The analytical solution of the model in mean field limit revealed that memories can be retrieved without any error up to a finite storage capacity of…
We investigated how the stability of macroscopic states in the associative memory model is affected by synaptic depression. To this model, we applied the dynamical mean-field theory, which has recently been developed in stochastic neural…
Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits, yet classical results, such as Hopfield's model of associative memory, rely on symmetric connectivity that…
Dense Associative Memories or modern Hopfield networks permit storage and reliable retrieval of an exponentially large (in the dimension of feature space) number of memories. At the same time, their naive implementation is non-biological,…
Continuous-Time Recurrent Neural Networks (CTRNNs) have been widely used for their capacity to model complex temporal behaviour. However, their internal dynamics often remain difficult to interpret. In this paper, we propose a new class of…
We consider a generalization of the Hopfield model, where the entries of patterns are Gaussian and diluted. We focus on the high-storage regime and we investigate analytically the topological properties of the emergent network, as well as…
We analyze continuous Hopfield associative memories augmented by additional, rapid short-term associative synaptic plasticity. Through the cavity method, we determine the boundary between the retrieval and forgetting, or spin-glass phase,…
We study Hopfield networks with non-reciprocal coupling inducing switches between memory patterns. Dynamical phase transitions occur between phases of no memory retrieval, retrieval of multiple point-attractors, and limit-cycle attractors.…
We solve the dynamics of Hopfield-type neural networks which store sequences of patterns, close to saturation. The asymmetry of the interaction matrix in such models leads to violation of detailed balance, ruling out an equilibrium…