Related papers: Beyond catastrophic forgetting in associative netw…

Catastrophic forgetting remains a central challenge in continual learning (CL), yet lacks a unified information-theoretic explanation for why some architectures forget catastrophically while others do not. We introduce \emph{Context Channel…

Attractor neural networks (ANNs) are one of the leading theoretical frameworks for the formation and retrieval of memories in networks of biological neurons. In this framework, a pattern imposed by external inputs to the network is said to…

Hippocampal CA3 is crucial for the formation of long-term associative memory. It has a heavily recurrent connectivity, and memories are thought to be stored as memory engrams in the CA3. However, despite its importance for memory storage…

Attractor neural network is an important theoretical scenario for modeling memory function in the hippocampus and in the cortex. In these models, memories are stored in the plastic recurrent connections of neural populations in the form of…

We introduce a systematic method for expanding general spin-glass Hamiltonians in terms of Mattis interactions, providing a novel perspective for understanding the fundamental differences between short-range Edwards-Anderson (EA) and…

Associative memory architectures such as the Hopfield network have long been important conceptual and theoretical models for neuroscience and artificial intelligence. However, translating these abstract models into spiking neural networks…

Humans can learn in a continuous manner. Old rarely utilized knowledge can be overwritten by new incoming information while important, frequently used knowledge is prevented from being erased. In artificial learning systems, lifelong…

In realistic neural circuits, both neurons and synapses are coupled in dynamics with separate time scales. The circuit functions are intimately related to these coupled dynamics. However, it remains challenging to understand the intrinsic…

The idea of using metaplastic synapses to incorporate the separate storage of long- and short-term memories via an array of hidden states was put forward in the cascade model of Fusi et al. In this paper, we devise and investigate two…

Biological synapses effortlessly balance memory retention and flexibility, yet artificial neural networks still struggle with the extremes of catastrophic forgetting and catastrophic remembering. Here, we introduce Metaplasticity from…

Emergent behavior in active systems is a complex byproduct of local, often pairwise, interactions. One such interaction is self-avoidance, which experimentally can arise as a response to self-generated environmental signals; such…

Retrieval of episodic memory is a dynamical process in the large scale brain networks. In social groups, the neural patterns, associated to specific events directly experienced by single members, are encoded, recalled and shared by all…

In neural circuits, synaptic strengths influence neuronal activity by shaping network dynamics, and neuronal activity influences synaptic strengths through activity-dependent plasticity. Motivated by this fact, we study a recurrent-network…

Continual acquisition of novel experience without interfering previously learned knowledge, i.e. continual learning, is critical for artificial neural networks, but limited by catastrophic forgetting. A neural network adjusts its parameters…

Uncovering the mechanisms behind long-term memory is one of the most fascinating open problems in neuroscience and artificial intelligence. Artificial associative memory networks have been used to formalize important aspects of biological…

Recently a daily routine for associative neural networks has been proposed: the network Hebbian-learns during the awake state (thus behaving as a standard Hopfield model), then, during its sleep state, optimizing information storage, it…

Humans learn and form memories in stochastic environments. Auto-associative memory systems model these processes by storing patterns and later recovering them from corrupted versions. Here, memories are learned by associating each pattern…

A spontaneously active neural system that is capable of continual learning should also be capable of homeostasis of both firing rate and connectivity. Experimental evidence suggests that both types of homeostasis exist, and that…

The organizational principles behind the connectivity of a complex network are known to influence its behavior. In this work we investigate, using the Hopfield model, the influence of the network architecture on the performance for…

Hopfield neural networks are a possible basis for modelling associative memory in living organisms. After summarising previous studies in the field, we take a new look at learning rules, exhibiting them as descent-type algorithms for…