Related papers: On-device Synaptic Memory Consolidation using Fowl…
Contrastive learning has significantly improved representation quality, enhancing knowledge transfer across tasks in continual learning (CL). However, catastrophic forgetting remains a key challenge, as contrastive based methods primarily…
We develop hybrid memory architectures for general-purpose sequence processing neural networks, that combine key-value memory using softmax attention (KV-memory) with fast weight memory through dynamic synaptic modulation (FW-memory) -- the…
Convolutional Neural Networks (CNNs) are one of the most successful deep machine learning technologies for processing image, voice and video data. CNNs require large amounts of processing capacity and memory, which can exceed the resources…
We numerically test an optimization method for deep neural networks (DNNs) using quantum fluctuations inspired by quantum annealing. For efficient optimization, our method utilizes the quantum tunneling effect beyond the potential barriers.…
Neuromorphic computing has emerged as a promising avenue towards building the next generation of intelligent computing systems. It has been proposed that memristive devices, which exhibit history-dependent conductivity modulation, could…
It is known that storage capacity per synapse increases by synaptic pruning in the case of a correlation-type associative memory model. However, the storage capacity of the entire network then decreases. To overcome this difficulty, we…
CMOS-compatible HfO2-based ferroelectric tunnel junction (FTJ) has attracted significant attention as a promising candidate for in-memory computing (IMC) due to its extremely low power consumption. However, conventional FTJs face inherent…
A hallmark of biological intelligence is neural reuse,the ability to preserve past learning and repurpose it for new tasks and changing environments. Photonic neural hardware offers high-bandwidth, low-latency computation, but current…
Replicating the computational functionalities and performances of the brain remains one of the biggest challenges for the future of information and communication technologies. Such an ambitious goal requires research efforts from the…
Convolutional neural networks (CNNs) have gained tremendous success in solving complex inverse problems. The aim of this work is to develop a novel CNN framework to reconstruct video sequence of dynamic live cells captured using a…
Spiking recurrent neural networks (RNNs) are a promising tool for solving a wide variety of complex cognitive and motor tasks, due to their rich temporal dynamics and sparse processing. However training spiking RNNs on dedicated…
In neuromorphic photonic systems, device operations are typically governed by analog signals, necessitating digital-to-analog converters (DAC) and analog-to-digital converters (ADC). However, data movement between memory and these…
The ability to learn in dynamic, nonstationary environments without forgetting previous knowledge, also known as Continual Learning (CL), is a key enabler for scalable and trustworthy deployments of adaptive solutions. While the importance…
Convolutional neural networks are state-of-the-art and ubiquitous in modern signal processing and machine vision. Nowadays, hardware solutions based on emerging nanodevices are designed to reduce the power consumption of these networks.…
The human brain achieves exceptional energy efficiency by co-locating memory and processing, yet reproducing this principle in hardware remains challenging because many neuromorphic devices require standby power, offer limited…
Fully analogue neural computation requires hardware that can implement both linear and nonlinear transformations without digital assistance. While analogue in-memory computing efficiently realizes matrix-vector multiplication, the absence…
This paper is devoted to the features of the practical application of the Elastic Weight Consolidation (EWC) method for continual learning of neural networks on several training sets. We will more rigorously compare the well-known…
Working memory -- the ability to store and recall precise temporal patterns of neural activity -- remains an open challenge for spiking neural networks (SNNs). We propose a recurrent SNN of $N$ neurons in which each synapse is equipped with…
Brain-inspired computation promises complex cognitive tasks at biological energy efficiencies. The brain contains $10^4$ synapses per neuron. Hence, ultra-low energy, high-density synapses are needed for spiking neural networks (SNN). In…
The storage of continuous variables in working memory is hypothesized to be sustained in the brain by the dynamics of recurrent neural networks (RNNs) whose steady states form continuous manifolds. In some cases, it is thought that the…