Related papers: Amorphous Solid Model of Vectorial Hopfield Neural…
Recent research has established a connection between modern Hopfield networks (HNs) and transformer attention heads, with guarantees of exponential storage capacity. However, these models still face challenges scaling storage efficiently.…
Associative memories in the brain receive and store patterns of activity registered by the sensory neurons, and are able to retrieve them when necessary. Due to their importance in human intelligence, computational models of associative…
Reasoning is the ability to integrate internal states and external inputs in a meaningful and semantically consistent flow. Contemporary machine learning (ML) systems increasingly rely on such sequential reasoning, from language…
Recent works have highlighted the need for a new dynamical paradigm in the modeling of brain function and evolution. Specifically, these models should incorporate non-constant and asymmetric synaptic weights $T_{ij}$ in the neuron-neuron…
The slowing of Moore's law and the increasing energy demands of machine learning present critical challenges for both the hardware and machine learning communities, and drive the development of novel computing paradigms. Of particular…
The set of the fixed points of the Hopfield type network is under investigation. The connection matrix of the network is constructed according to the Hebb rule from the set of memorized patterns which are treated as distorted copies of the…
Vector Symbolic Architectures (VSAs) have emerged as a novel framework for enabling interpretable machine learning algorithms equipped with the ability to reason and explain their decision processes. The basic idea is to represent discrete…
Network systems can exhibit memory effects in which the interactions between different pairs of nodes adapt in time, leading to the emergence of preferred connections, patterns, and sub-networks. To a first approximation, this memory can be…
Among the performance-enhancing procedures for Hopfield-type networks that implement associative memory, Hebbian Unlearning (or dreaming) strikes for its simplicity and its clear biological interpretation. Yet, it does not easily lend…
We study a neural network model in which both neurons and synaptic interactions evolve in time simultaneously. The time evolution of synaptic interactions is described by a Langevin equation including a Hebbian learning term, and a bias…
The aim of the present paper is to study the effects of Hebbian learning in random recurrent neural networks with biological connectivity, i.e. sparse connections and separate populations of excitatory and inhibitory neurons. We furthermore…
In the past three decades, many theoretical measures of complexity have been proposed to help understand complex systems. In this work, for the first time, we place these measures on a level playing field, to explore the qualitative…
Recently, the original storage prescription for the Hopfield model of neural networks -- as well as for its dense generalizations -- has been turned into a genuine Hebbian learning rule by postulating the expression of its Hamiltonian for…
We solve the dynamics of Hopfield-type neural networks which store sequences of patterns, close to saturation. The asymmetry of the interaction matrix in such models leads to violation of detailed balance, ruling out an equilibrium…
We propose a modification of the cost function of the Hopfield model whose salient features shine in its Taylor expansion and result in more than pairwise interactions with alternate signs, suggesting a unified framework for handling both…
We introduce a modern Hopfield network with continuous states and a corresponding update rule. The new Hopfield network can store exponentially (with the dimension of the associative space) many patterns, retrieves the pattern with one…
The modern Hopfield network, proposed by Krotov and Hopfield, is a mathematical generalization of the Hopfield network, which is a basic model of associative memory that employs higher-order interactions. This study introduces an open…
A specific type of neural network, the Restricted Boltzmann Machine (RBM), is implemented for classification and feature detection in machine learning. RBM is characterized by separate layers of visible and hidden units, which are able to…
Attractor neural network is an important theoretical scenario for modeling memory function in the hippocampus and in the cortex. In these models, memories are stored in the plastic recurrent connections of neural populations in the form of…
The set of the fixed points of the Hopfield type network is under investigation. The connection matrix of the network is constructed according the Hebb rule from the set of memorized patterns which are treated as distorted copies of the…