Related papers: Temporal pattern recognition with delayed feedback…
In vision-based action recognition, spatio-temporal features from different modalities are used for recognizing activities. Temporal modeling is a long challenge of action recognition. However, there are limited methods such as pre-computed…
Time perception is essential for task switching, and in the mammalian brain appears alongside other processes. Memristors are electronic components used as synapses and as models for neurons. The d.c. response of memristors can be…
Due to simplicity and strong stability guarantees, predictor feedback methods have stood as a popular approach for time delay systems since the 1950s. For time-varying delays, however, implementation requires computing a prediction horizon…
Neural networks powered with external memory simulate computer behaviors. These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks. In this paper, we introduce a new memory to…
The influence of time delay in systems of two coupled excitable neurons is studied in the framework of the FitzHugh-Nagumo model. Time-delay can occur in the coupling between neurons or in a self-feedback loop. The stochastic…
Pre-trained language models demonstrate general intelligence and common sense, but long inputs quickly become a bottleneck for memorizing information at inference time. We resurface a simple method, Memorizing Transformers (Wu et al.,…
Schemes of switching nanomagnetic memories via the effect of spin torque with various polarizations of injected electrons are studied. Simulations based on macrospin and micromagnetic theories are performed and compared. We demonstrate that…
Memory effects are ubiquitous in nature and are particularly relevant at the nanoscale where the dynamical properties of electrons and ions strongly depend on the history of the system, at least within certain time scales. We review here…
The dynamics of vortex based spin-torque nano-oscillators is investigated theoretically. Starting from a fully analytical model based on the Thiele equation approach, fine-tuned data-driven corrections are carried out to the gyrotropic and…
Entropic uncertainty relations are universal quantifiers of fundamental uncertainties of quantum measurements and are widely discussed in the quantum metrology literature. Quantum memory is a phenomenon related to the specific type of…
We present a conformal prediction method for time series using the Transformer architecture to capture long-memory and long-range dependencies. Specifically, we use the Transformer decoder as a conditional quantile estimator to predict the…
As nascent nonlinear oscillators, nano-constriction spin Hall nano-oscillators (SHNOs) represent a promising potential for integration into more complicated systems such as neural networks, magnetic field sensors, and radio frequency (RF)…
Transformers have been successfully applied to sequential, auto-regressive tasks despite being feedforward networks. Unlike recurrent neural networks, Transformers use attention to capture temporal relations while processing input tokens in…
Photonic neuromorphic computing offers compelling advantages in power efficiency and parallel processing, but often falls short in realizing scalable nonlinearity and long-term memory. We overcome these limitations by employing silicon…
Animal sensorimotor behavior is frequently modeled using optimal controllers. However, it is unclear how the neural circuits within the animal's nervous system implement optimal controller-like behavior. In this work, we study the question…
The spiking neural network (SNN) mimics the information processing operation in the human brain, represents and transmits information in spike trains containing wealthy spatial and temporal information, and shows superior performance on…
Spiking neural network (SNN) is interesting both theoretically and practically because of its strong bio-inspiration nature and potentially outstanding energy efficiency. Unfortunately, its development has fallen far behind the conventional…
Spiking Neural Networks (SNNs), as one of the algorithmic models in neuromorphic computing, have gained a great deal of research attention owing to temporal information processing capability, low power consumption, and high biological…
Transformers have achieved remarkable success in sequence modeling and beyond but suffer from quadratic computational and memory complexities with respect to the length of the input sequence. Leveraging techniques include sparse and linear…
The Transformer architecture has revolutionized deep learning on sequential data, becoming ubiquitous in state-of-the-art solutions for a wide variety of applications. Yet vanilla Transformers are notoriously resource-expensive, requiring…