Related papers: Memory Recall and Spike Frequency Adaptation
Spin-glass models of associative memories are a cornerstone between statistical physics and theoretical neuroscience. In these networks, stochastic spin-like units interact through a synaptic matrix shaped by local Hebbian learning. In…
Learning to remember long sequences remains a challenging task for recurrent neural networks. Register memory and attention mechanisms were both proposed to resolve the issue with either high computational cost to retain memory…
Autoassociative networks were proposed in the 80's as simplified models of memory function in the brain, using recurrent connectivity with hebbian plasticity to store patterns of neural activity that can be later recalled. This type of…
It is shown that a Hopfield recurrent neural network exhibits a scaling regime, whose specific exponents depend on the number of parcels used and the decay length of the coupling strength. This scaling regime recovers the picture introduced…
Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly,…
In \cite{Hop82}, Hopfield introduced a \emph{Hebbian} learning rule based neural network model and suggested how it can efficiently operate as an associative memory. Studying random binary patterns, he also uncovered that, if a small…
Reasoning is the ability to integrate internal states and external inputs in a meaningful and semantically consistent flow. Contemporary machine learning (ML) systems increasingly rely on such sequential reasoning, from language…
The Self-Optimization (SO) model can be considered as the third operational mode of the classical Hopfield Network, leveraging the power of associative memory to enhance optimization performance. Moreover, it has been argued to express…
We study associative memory based on temporal coding in which successful retrieval is realized as an entrainment in a network of simple phase oscillators with distributed natural frequencies under the influence of white noise. The memory…
Statistical mechanics has made significant contributions to the study of biological neural systems by modeling them as recurrent networks of interconnected units with adjustable interactions. Several algorithms have been proposed to…
We discuss how inference can be performed when data are sampled from the non-ergodic phase of systems with multiple attractors. We take as model system the finite connectivity Hopfield model in the memory phase and suggest a cavity method…
Being hit by a ball is usually not a pleasant experience. While a ball may not be fatal, other objects can be. To protect themselves, many organisms, from humans to insects, have developed neuronal mechanisms to signal approaching objects…
Remembering and forgetting mechanisms are two sides of the same coin in a human learning-memory system. Inspired by human brain memory mechanisms, modern machine learning systems have been working to endow machine with lifelong learning…
We investigate a recurrent neural network model with common external and bias inputs that can retrieve branching sequences. Retrieval of memory sequences is one of the most important functions of the brain. A lot of research has been done…
We investigate a large ensemble of Quadratic Integrate-and-Fire (QIF) neurons with heterogeneous input currents and adaptation variables. Our analysis reveals that for a specific class of adaptation, termed quadratic spike-frequency…
Qualitatively, some real networks in the brain could be characterized as 'small worlds', in the sense that the structure of their connections is intermediate between the extremes of an orderly geometric arrangement and of a…
Topological data analyses are rapidly turning into key tools for quantifying large volumes of neurobiological data, e.g., for organizing the spiking outputs of large neuronal ensembles and thus gaining insights into the information produced…
Forming accurate memory of sequential stimuli is a fundamental function of biological agents. However, the computational mechanism underlying sequential memory in the brain remains unclear. Inspired by neuroscience theories and recent…
We analyze continuous Hopfield associative memories augmented by additional, rapid short-term associative synaptic plasticity. Through the cavity method, we determine the boundary between the retrieval and forgetting, or spin-glass phase,…
Based on recent work by Gripon and Berrou, we introduce a new model of an associative memory. We show that this model has an efficiency bounded away from 0 and is therefore significantly more effective than the well known Hopfield model. We…