Related papers: Memorization in a neural network with adjustable t…
We present emergent mechanical memory storage behavior in soft cellular materials. The cellular materials are a network of soft hyperelastic rods which store shape changes, specifically local indentation. This happens under an applied…
Sleep is thought to support memory consolidation and the recovery of optimal energetic regime by reorganizing synaptic connectivity, yet how plasticity across hierarchical brain circuits contributes to abstraction and energy efficiency…
Replicating the computational functionalities and performances of the brain remains one of the biggest challenges for the future of information and communication technologies. Such an ambitious goal requires research efforts from the…
Standard accounts of memory consolidation emphasise the stabilisation of stored representations, but struggle to explain representational drift, semanticisation, or the necessity of offline replay. Here we propose that high-capacity…
The inertia of trapping and detrapping of nonequilibrium charge carriers affects the electrochemical and transport properties of both bulk and nanoscopic structures in a very peculiar way. An emerging memory response with a hysteresis in…
In continual learning, plasticity refers to the ability of an agent to quickly adapt to new information. Neural networks are known to lose plasticity when processing non-stationary data streams. In this paper, we propose L2 Init, a simple…
Data often comes in the form of an array or matrix. Matrix factorization techniques attempt to recover missing or corrupted entries by assuming that the matrix can be written as the product of two low-rank matrices. In other words, matrix…
The ability of artificial agents to increment their capabilities when confronted with new data is an open challenge in artificial intelligence. The main challenge faced in such cases is catastrophic forgetting, i.e., the tendency of neural…
The plasticity of the conduction delay between neurons plays a fundamental role in learning. However, the exact underlying mechanisms in the brain for this modulation is still an open problem. Understanding the precise adjustment of…
Collective rhythmic dynamics from neurons is vital for cognitive functions such as memory formation but how neurons self-organize to produce such activity is not well understood. Attractor-based models have been successfully implemented as…
We introduce the percolation with plasticity (PWP) systems that exhibit neuromorphic functionalities including multi-valued memory, random number generation, matrix-vector multiplication, and associative learning. PWP systems have multiple…
We develop a statistical theory of temporal learnability in recurrent neural networks, quantifying the maximal temporal horizon $\mathcal{H}_N$ over which gradient-based learning can recover lag-dependent structure at finite sample size…
We develop a theory of percolation with plasticity systems (PWPs) rendering properties of interest for neuromorphic computing. Unlike the standard percolation between two large electrodes, they have multiple ($N\gg 1$) interfaces and…
The success of pre-trained contextualized representations has prompted researchers to analyze them for the presence of linguistic information. Indeed, it is natural to assume that these pre-trained representations do encode some level of…
We introduce Continual Learning via Neural Pruning (CLNP), a new method aimed at lifelong learning in fixed capacity models based on neuronal model sparsification. In this method, subsequent tasks are trained using the inactive neurons and…
The neural network memorization problem is to study the expressive power of neural networks to interpolate a finite dataset. Although memorization is widely believed to have a close relationship with the strong generalizability of deep…
Contextual memory integration remains a high challenge in the development of language models, particularly in tasks that require maintaining coherence over extended sequences. Traditional approaches, such as self-attention mechanisms and…
Synaptic plasticity, the dynamic tuning of signal transmission strength between neurons, serves as a fundamental basis for memory and learning in biological organisms. This adaptive nature of synapses is considered one of the key features…
Competition between synapses arises in some forms of correlation-based plasticity. Here we propose a game theory-inspired model of synaptic interactions whose dynamics is driven by competition between synapses in their weak and strong…
Humans and animals have the ability to continually acquire, fine-tune, and transfer knowledge and skills throughout their lifespan. This ability, referred to as lifelong learning, is mediated by a rich set of neurocognitive mechanisms that…