Related papers: Short-Term Memory Through Persistent Activity: Evo…
Many biological, social and man-made systems are better described in terms of temporal networks, i.e. networks whose links are only present at certain points in time, rather than by static ones. In particular, it has been found that…
To maximize future rewards in this ever-changing world, animals must be able to discover the temporal structure of stimuli and then anticipate or act correctly at the right time. How the animals perceive, maintain, and use time intervals…
Memory can be defined as the ability to retain and recall information in a diverse range of forms. It is a vital component of the way in which we as human beings operate on a day to day basis. Given a particular situation, decisions are…
Inhibitory neurons play a crucial role in maintaining persistent neuronal activity. Although connected extensively through electrical synapses (gap-junctions), these neurons also exhibit interactions through chemical synapses in certain…
Long short-term memory (LSTM) based acoustic modeling methods have recently been shown to give state-of-the-art performance on some speech recognition tasks. To achieve a further performance improvement, in this research, deep extensions on…
Many systems are modulated by unknown slow processes. This hinders analysis in highly non-linear systems, such as excitable systems. We show that for such systems, if the input matches the sparse `spiky' nature of the output, the spiking…
Topological techniques have become a popular tool for studying information flows in neural networks. In particular, simplicial homology theory is used to analyze how cognitive representations of space emerge from large conglomerates of…
Common wisdom indicates that to implement a Dynamical Memory with spiking neurons two ingredients are necessary: recurrence and a neuron population. Here we shall show that the second requirement is not needed. We shall demonstrate that…
In this work we study, analytically and employing Monte Carlo simulations, the influence of the competition between several activity-dependent synaptic processes, such as short-term synaptic facilitation and depression, on the maximum…
Natural spatiotemporal processes can be highly non-stationary in many ways, e.g. the low-level non-stationarity such as spatial correlations or temporal dependencies of local pixel values; and the high-level variations such as the…
Efficient pattern separation in dentate gyrus plays an important role in storing information in the hippocampus. Current knowledge of the structure and function of the hippocampus, entorhinal cortex and dentate gyrus, in pattern separation…
Memory Networks have emerged as effective models to incorporate Knowledge Bases (KB) into neural networks. By storing KB embeddings into a memory component, these models can learn meaningful representations that are grounded to external…
Even as machine learning exceeds human-level performance on many applications, the generality, robustness, and rapidity of the brain's learning capabilities remain unmatched. How cognition arises from neural activity is a central open…
While deep learning has led to remarkable advances across diverse applications, it struggles in domains where the data distribution changes over the course of learning. In stark contrast, biological neural networks continually adapt to…
Long short-term memory recurrent neural networks (LSTM-RNNs) are considered state-of-the art in many speech processing tasks. The recurrence in the network, in principle, allows any input to be remembered for an indefinite time, a feature…
Recurrent neural networks (RNNs) provide state-of-the-art performances in a wide variety of tasks that require memory. These performances can often be achieved thanks to gated recurrent cells such as gated recurrent units (GRU) and long…
One of the defining features of living systems is their adaptability to changing environmental conditions. This requires organisms to extract temporal and spatial features of their environment, and use that information to compute the…
Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions are of interests to engineers, mathematicians, and physicists from the last several decades. With a motivation of…
Hopfield neural networks are a possible basis for modelling associative memory in living organisms. After summarising previous studies in the field, we take a new look at learning rules, exhibiting them as descent-type algorithms for…
Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and Memory Networks which contain memory are popularly used to learn patterns in sequential data. Sequential data has long sequences that hold relationships. RNN can…