Related papers: Memorization in a neural network with adjustable t…

We introduce a new method for internal replay that modulates the frequency of rehearsal based on the depth of the network. While replay strategies mitigate the effects of catastrophic forgetting in neural networks, recent works on…

The subject of this paper is the evolution of the concept of information processing in regular structures based on multi-level processing in nested cellular automata. The essence of the proposed model is a discrete space-time containing…

Contrastive representation learning has emerged as a promising technique for continual learning as it can learn representations that are robust to catastrophic forgetting and generalize well to unseen future tasks. Previous work in…

The rapid scaling of artificial neural networks has exposed fundamental limitations of conventional von Neumann computing architectures. In these systems, the physical separation between memory and processing creates a bottleneck, as…

Despite remarkable successes achieved by modern neural networks in a wide range of applications, these networks perform best in domain-specific stationary environments where they are trained only once on large-scale controlled data…

Objective. Long-term potentiation (LTP) is a fundamental mechanism underlying learning and memory, yet its investigation at the network level in vitro remains challenging, particularly when optogenetic stimulation is used. The objective of…

This paper studies how the model architecture and data configurations influence the empirical memorization capacity of generative transformers. The models are trained using synthetic text datasets derived from the Systematized Nomenclature…

Neural networks are susceptible to catastrophic forgetting. They fail to preserve previously acquired knowledge when adapting to new tasks. Inspired by human associative memory system, we propose a brain-like approach that imitates the…

Catastrophic forgetting is a major problem in continual learning, and lots of approaches arise to reduce it. However, most of them are evaluated through task accuracy, which ignores the internal model structure. Recent research suggests…

The task of a neural associative memory is to retrieve a set of previously memorized patterns from their noisy versions using a network of neurons. An ideal network should have the ability to 1) learn a set of patterns as they arrive, 2)…

We simulate static memory materials on a two-dimensional lattice. The bulk properties of such materials depend on boundary conditions. Considerable information can be stored in various local patterns. We observe local probabilities…

Inferring temporal interaction graphs and higher-order structure from neural signals is a key problem in building generative models for systems neuroscience. Foundation models for large-scale neural data represent shared latent structures…

This study presents the applicability of conventional deep recurrent neural networks (RNN) to predict path-dependent plasticity associated with material heterogeneity and anisotropy. Although the architecture of RNN possesses inductive…

Understanding the basic operational logics of the nervous system is essential to advancing neuroscientific research. However, theoretical efforts to tackle this fundamental problem are lacking, despite the abundant empirical data about the…

The ability to flexibly compose previously acquired skills to execute intelligent behaviors is a hallmark of natural intelligence. Such compositional flexibility is often attributed to context-dependent gating mechanisms that determine how…

Natural memories are associative, declarative and distributed. Symbolic computing memories resemble natural memories in their declarative character, and information can be stored and recovered explicitly; however, they lack the associative…

Continual learning (CL) is a technique that enables neural networks to constantly adapt to their dynamic surroundings. Despite being overlooked for a long time, this technology can considerably address the customized needs of users in edge…

The idea of using metaplastic synapses to incorporate the separate storage of long- and short-term memories via an array of hidden states was put forward in the cascade model of Fusi et al. In this paper, we devise and investigate two…

In long-term potentiation (LTP), one of the most studied types of neural plasticity, synaptic strength is persistently increased in response to stimulation. Although a number of different proteins have been implicated in the sub-cellular…

Associative learning is one of the key mechanisms displayed by living organisms in order to adapt to their changing environments. It was early recognized to be a general trait of complex multicellular organisms but also found in "simpler"…