Related papers: How glassy are neural networks?
In this paper we study, via equilibrium statistical mechanics, the properties of the internal energy of an Hopfield neural network whose patterns are stored continuously (Gaussian distributed). The model is shown to be equivalent to a…
The Hopfield model describes a neural network that stores memories using all-to-all-coupled spins. Memory patterns are recalled under equilibrium dynamics. Storing too many patterns breaks the associative recall process because frustration…
Understanding the glassy nature of neural networks is pivotal both for theoretical and computational advances in Machine Learning and Theoretical Artificial Intelligence. Keeping the focus on dense associative Hebbian neural networks, the…
Spin glasses, generally defined as disordered systems with randomized competing interactions, are a widely investigated complex system. Theoretical models describing spin glasses are broadly used in other complex systems, such as those…
We study a class of Hopfield models where the memories are represented by a mixture of Gaussian and binary variables and the neurons are Ising spins. We study the properties of this family of models as the relative weight of the two kinds…
We consider a generalization of the Hopfield model, where the entries of patterns are Gaussian and diluted. We focus on the high-storage regime and we investigate analytically the topological properties of the emergent network, as well as…
The spherical version of the Hopfield model for pattern recognition is considered in the static limit. Structures inside the patterns are modeled by Gaussian random variables that reward correlation between pairs of spins in a given…
We use belief-propagation techniques to study the equilibrium behavior of a bipartite spin-glass, with interactions between two sets of $N$ and $P = \alpha N$ spins. Each spin has a finite degree, i.e.\ number of interaction partners in the…
We present results for two different kinds of high order connections between neurons acting as corrections to the Hopfield model. Equilibrium properties are analyzed using the replica mean-field theory and compared with numerical…
Quantum neural networks form one pillar of the emergent field of quantum machine learning. Here, quantum generalisations of classical networks realizing associative memories - capable of retrieving patterns, or memories, from corrupted…
In this paper we continue our investigation of the analogical neural network, paying interest to its replica symmetric behavior in the absence of external fields of any type. Bridging the neural network to a bipartite spin-glass, we…
In the last five decades, mean-field neural-networks have played a crucial role in modelling associative memories and, in particular, the Hopfield model has been extensively studied using tools borrowed from the statistical mechanics of…
We investigate thermodynamic phase transitions of the joint presence of spin glass (SG) and random field (RF) using a random graph model that allows us to deal with the quenched disorder. Therefore, the connectivity becomes a controllable…
This work presents a statistical mechanics characterization of neural networks, motivated by the replica symmetry breaking (RSB) phenomenon in spin glasses. A Hopfield-type spin glass model is constructed from a given feedforward neural…
We develop a statistical mechanical approach based on the replica method to study the design space of deep and wide neural networks constrained to meet a large number of training data. Specifically, we analyze the configuration space of the…
Pattern storage by a single neuron is revisited. Generalizing Parisi's framework for spin glasses we obtain a variational free energy functional for the neuron. The solution is demonstrated at high temperature and large relative number of…
The classical result of concentration of the Gaussian measure on the sphere in the limit of large dimension induces a natural duality between Gaussian and spherical models of spin glass. We analyse the Legendre variational structure linking…
In this work we consider the {\em analog bipartite spin-glass} (or {\em real-valued restricted Boltzmann machine} in a neural network jargon), whose variables (those quenched as well as those dynamical) share standard Gaussian…
Recently a daily routine for associative neural networks has been proposed: the network Hebbian-learns during the awake state (thus behaving as a standard Hopfield model), then, during its sleep state, optimizing information storage, it…
In this work we develop analytical techniques to investigate a broad class of associative neural networks set in the high-storage regime. These techniques translate the original statistical-mechanical problem into an analytical-mechanical…