Related papers: Recognition Capabilities of a Hopfield Model with …
The Hopfield model is a pioneering neural network model with associative memory retrieval. The analytical solution of the model in mean field limit revealed that memories can be retrieved without any error up to a finite storage capacity of…
We propose and analyze a new variation of the so-called {\em exponential Hopfield model}, a recently introduced family of associative neural networks with unprecedented storage capacity. Our construction is based on a cost function defined…
Dense Associative Memories or Modern Hopfield Networks have many appealing properties of associative memory. They can do pattern completion, store a large number of memories, and can be described using a recurrent neural network with a…
Unsupervised learning in a generalized Hopfield associative-memory network is investigated in this work. First, we prove that the (generalized) Hopfield model is equivalent to a semi-restricted Boltzmann machine with a layer of visible…
Energy-based probabilistic models learned by maximizing the likelihood of the data are limited by the intractability of the partition function. A widely used workaround is to maximize the pseudo-likelihood, which replaces the global…
It has been recently shown that a learning transition happens when a Hopfield Network stores examples generated as superpositions of random features, where new attractors corresponding to such features appear in the model. In this work we…
The Hopfield model is a paradigmatic model of neural networks that has been analyzed for many decades in the statistical physics, neuroscience, and machine learning communities. Inspired by the manifold hypothesis in machine learning, we…
Dense Associative Memories or modern Hopfield networks permit storage and reliable retrieval of an exponentially large (in the dimension of feature space) number of memories. At the same time, their naive implementation is non-biological,…
The Hebbian unlearning algorithm, i.e. an unsupervised local procedure used to improve the retrieval properties in Hopfield-like neural networks, is numerically compared to a supervised algorithm to train a linear symmetric perceptron. We…
We studied the computational properties of an attractor neural network (ANN) with different network topologies. Though fully connected neural networks exhibit, in general, a good performance, they are biologically unrealistic, as it is…
We analyze a variant of the Hopfield model that incorporates an unlearning mechanism based on spin correlations in the high-temperature regime. In the large system limit where extensively many patterns are stored, we employ the replica…
We present a nonparametric interpretation for deep learning compatible modern Hopfield models and utilize this new perspective to debut efficient variants. Our key contribution stems from interpreting the memory storage and retrieval…
This article delves into the Hopfield neural network model, drawing inspiration from biological neural systems. The exploration begins with an overview of the model's foundations, incorporating insights from mechanical statistics to deepen…
We present results for two different kinds of high order connections between neurons acting as corrections to the Hopfield model. Equilibrium properties are analyzed using the replica mean-field theory and compared with numerical…
Associative memory, a form of content-addressable memory, facilitates information storage and retrieval in many biological and physical systems. In statistical mechanics models, associative memory at equilibrium is represented through…
High-order extensions of the Hopfield model are known to exhibit dramatically enhanced storage capacity at equilibrium, while their dynamical retrieval properties remain less well understood. In our previous work, we carried out a dynamical…
Associative memory models are content-addressable memory systems fundamental to biological intelligence and are notable for their high interpretability. However, existing models evaluate the quality of retrieval based on proximity, which…
Dense associative memory, a fundamental instance of modern Hopfield networks, can store a large number of memory patterns as equilibrium states of recurrent networks. While the stationary-state storage capacity has been investigated, its…
This paper presents a new artificial neuron model capable of learning its receptive field in the topological domain of inputs. The model provides adaptive and differentiable local connectivity (plasticity) applicable to any domain. It…
We introduce an Outlier-Efficient Modern Hopfield Model (termed $\mathrm{OutEffHop}$) and use it to address the outlier inefficiency problem of {training} gigantic transformer-based models. Our main contribution is a novel associative…