Related papers: High storage capacity in the Hopfield model with a…
Memory and forgetting constitute two sides of the same coin, and although the first has been rigorously investigated, the latter is often overlooked. A number of experiments under the realm of psychology and experimental neuroscience have…
We discuss the properties of equilibrium states in an autoassociative memory model storing hierarchically correlated patterns (hereafter, hierarchical patterns). We will show that symmetric mixed states (hereafter, mixed states) are…
The dynamical behavior of switched affine systems is known to be more intricate than that of the well-studied switched linear systems, essentially due to the existence of distinct equilibrium points for each subsystem. First, under…
Time-dependent data-generating distributions have proven to be difficult for gradient-based training of neural networks, as the greedy updates result in catastrophic forgetting of previously learned knowledge. Despite the progress in the…
A naive model of many networked logistic maps with an excitation type coupling [Neural Networks, vol. 20, 102--108 (2007)], which is an extension of other low dimensional models, has been recently proposed to mimic the waking-sleeping…
This work focuses on the training dynamics of one associative memory module storing outer products of token embeddings. We reduce this problem to the study of a system of particles, which interact according to properties of the data…
This paper generalizes the existing minimal model of the hypothalamic-pituitary-adrenal (HPA) axis in a realistic way, by including memory terms: distributed time delays, on one hand and fractional-order derivatives, on the other hand. The…
Current deep learning models often suffer from catastrophic forgetting of old knowledge when continually learning new knowledge. Existing strategies to alleviate this issue often fix the trade-off between keeping old knowledge (stability)…
In this paper, we investigate the associative memory in recurrent neural networks, based on the model of evolving neural networks proposed by Nolfi, Miglino and Parisi. Experimentally developed network has highly asymmetric synaptic weights…
Adaptive optimal control using value iteration initiated from a stabilizing control policy is theoretically analyzed in terms of stability of the system during the learning stage without ignoring the effects of approximation errors. This…
The behavior of the network and its stability are governed by both dynamics of individual nodes as well as their topological interconnections. Attention mechanism as an integral part of neural network models was initially designed for…
We study the synchronous dynamics of the Hopfield model when a random antisymmetric part is added to the otherwise symmetric synaptic matrix. We use a generating functional technique to derive analytical expressions for the order parameters…
The spherical version of the Hopfield model for pattern recognition is considered in the static limit. Structures inside the patterns are modeled by Gaussian random variables that reward correlation between pairs of spins in a given…
The Self-Optimization (SO) model can be considered as the third operational mode of the classical Hopfield Network, leveraging the power of associative memory to enhance optimization performance. Moreover, it has been argued to express…
To improve the storage capacity of the Hopfield model, we develop a version of the dreaming algorithm that perpetually reinforces the patterns to be stored (as in the Hebb rule), and erases the spurious memories (as in dreaming algorithms).…
The central question that we address is: How can structured information be stored in a hierarchical Hopfield model involving hidden layers? To this end, we develop a formalism of strokes and concepts that allows us to appropriately…
Survival models capture the relationship between an accumulating hazard and the occurrence of a singular event stimulated by that accumulation. When the model for the hazard is sufficiently flexible survival models can accommodate a wide…
We study both analytically and numerically the effects of including refractory periods in the Hopfield model for associative memory. These periods are introduced in the dynamics of the network as thresholds that depend on the state of the…
A stochastic model of excitatory and inhibitory interactions which bears universality traits is introduced and studied. The endogenous component of noise, stemming from finite size corrections, drives robust inter-nodes correlations, that…
Algorithmic stability is a central concept in statistics and learning theory that measures how sensitive an algorithm's output is to small changes in the training data. Stability plays a crucial role in understanding generalization,…