Related papers: Emergent Self-Attention from Astrocyte-Gated Assoc…
In this paper, we investigate convergence of a class of analytic neural networks with event-triggered rule. This model is general and include Hopfield neural network as a special case. The event-trigger rule efficiently reduces the…
Attractor networks are an influential theory for memory storage in brain systems. This theory has recently been challenged by the observation of strong temporal variability in neuronal recordings during memory tasks. In this work, we study…
We consider the class of Hopfield models of associative memory with activation function $F$ and state space $\{-1,1\}^N$, where each vertex of the cube describes a configuration of $N$ binary neurons. $M$ randomly chosen configurations,…
The standard consolidation theory states that short-term memories located in the hippocampus enable the consolidation of long-term memories in the neocortex. In other words, the neocortex slowly learns long-term memories with a transient…
We characterize the pre-softmax attention matrix $\mathbf{QK^\top}$ in transformers as an associative memory matrix encoding pairwise associations between input features. By decomposing this matrix into its symmetric and skew-symmetric…
We study a neural network model in which both neurons and synaptic interactions evolve in time simultaneously. The time evolution of synaptic interactions is described by a Langevin equation including a Hebbian learning term, and a bias…
Recent advances at the intersection of control theory, neuroscience, and machine learning have revealed novel mechanisms by which dynamical systems perform computation. These advances encompass a wide range of conceptual, mathematical, and…
Asymptotic synchronization is one of the essential differences between artificial neural networks and biologically inspired neural networks due to mismatches from dynamical update of weight parameters and heterogeneous activations. In this…
Models of interacting complex systems provide the fundamental statistical physics reference frame for the study and the understanding of associative memories, machine learning, and the dynamics of neural networks. On the other hand,…
Storing memory for molecular recognition is an efficient strategy for responding to external stimuli. Biological processes use different strategies to store memory. In the olfactory cortex, synaptic connections form when stimulated by an…
Learning and memory are intertwined in our brain and their relationship is at the core of several recent neural network models. In particular, the Attention-Gated MEmory Tagging model (AuGMEnT) is a reinforcement learning network with an…
In this paper we study robust synchronization of time-fractional Hopfield neural networks with memristive couplings and Hebbian learning rules. This new model of artificial neural networks exhibits strong memory and long-range…
In neuroscience, classical Hopfield networks are the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, memory-augmented neural networks in…
Hybrid architectures combining state-space models with attention have achieved strong efficiency-quality tradeoffs, yet existing approaches either apply attention uniformly or learn static sparse patterns. This misses a key opportunity:…
We investigate the asymptotic behavior of probability measures associated with stochastic dynamical systems featuring either globally contracting or $B_{r}$-contracting drift terms. While classical results often assume constant diffusion…
Learning or memory formation are associated with the strengthening of the synaptic connections between neurons according to a pattern reflected by the input. According to this theory a retained memory sequence is associated to a dynamic…
Many tasks require flexibly modifying perception and behavior based on current goals. Humans can retrieve episodic memories from days to years ago, using them to contextualize and generalize behaviors across novel but structurally related…
Continuous-Time Recurrent Neural Networks (CTRNNs) have been widely used for their capacity to model complex temporal behaviour. However, their internal dynamics often remain difficult to interpret. In this paper, we propose a new class of…
The Hopfield network (HN) is a classical model of associative memory whose dynamics are closely related to the Ising spin system with 2-body interactions. Stored patterns are encoded as minima of an energy function shaped by a Hebbian…
We introduce a spherical Hopfield-type neural network involving neurons and patterns that are continuous variables. We study both the thermodynamics and dynamics of this model. In order to have a retrieval phase a quartic term is added to…