Related papers: Replica Symmetry Breaking in Attractor Neural Netw…
I consider branches of Replica-Symmetry-Breaking (RSB) solutions in Glassy systems that display a dynamical transition at a temperature $T_d$ characterized by a Mode-Coupling-Theory dynamical behavior. Below $T_d$ these branches of…
We present results for two different kinds of high order connections between neurons acting as corrections to the Hopfield model. Equilibrium properties are analyzed using the replica mean-field theory and compared with numerical…
Hybrid architectures combining state-space models with attention have achieved strong efficiency-quality tradeoffs, yet existing approaches either apply attention uniformly or learn static sparse patterns. This misses a key opportunity:…
We investigate dense higher-order associative memories in the high storage regime when the stored patterns are biased, namely when the entries of the patterns are not symmetrically distributed around zero. In this setting, the standard…
Increasing the size of a Transformer does not always lead to enhanced performance. This phenomenon cannot be explained by the empirical scaling laws. Furthermore, the model's enhanced performance is closely associated with its memorization…
Sequence memory is an essential attribute of natural and artificial intelligence that enables agents to encode, store, and retrieve complex sequences of stimuli and actions. Computational models of sequence memory have been proposed where…
The Hopfield model provides a paradigmatic framework for associative memory. Its classical implementation, based on the Hebbian learning rule, suffers from catastrophic forgetting: when one attempts storing too many patterns, the network…
Cyclic patterns of neuronal activity are ubiquitous in animal nervous systems, and partially responsible for generating and controlling rhythmic movements such as locomotion, respiration, swallowing and so on. Clarifying the role of the…
Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for…
High-capacity associative memories based on Kernel Logistic Regression (KLR) exhibit strong storage capabilities, but the dynamical and geometric mechanisms underlying their stability remain poorly understood. This paper investigates the…
Dense Associative Memories or modern Hopfield networks permit storage and reliable retrieval of an exponentially large (in the dimension of feature space) number of memories. At the same time, their naive implementation is non-biological,…
We report results of the analysis of the spontaneous symmetry breaking (SSB) in the basic (actually, simplest) model which is capable to produce the SSB phenomenology in the one-dimensional setting. It is based on the Gross-Pitaevskii -…
A new powerful method to test the stability of the replica symmetric spin glass phase is proposed by introducing a replicon generator function g(v). Exact symmetry arguments are used to prove that its extremum is proportional to the inverse…
We study the maximum-average submatrix problem, in which given an $N \times N$ matrix $J$ one needs to find the $k \times k$ submatrix with the largest average of entries. We study the problem for random matrices $J$ whose entries are…
We consider the class of Hopfield models of associative memory with activation function $F$ and state space $\{-1,1\}^N$, where each vertex of the cube describes a configuration of $N$ binary neurons. $M$ randomly chosen configurations,…
Simulational studies of spin glasses in the last decade have focussed on the so-called replicon exponent $\alpha$ as a means of determining whether the low-temperature phase of spin glasses is described by the replica symmetry breaking…
We consider a multi-layer Sherrington-Kirkpatrick spin-glass as a model for deep restricted Boltzmann machines and we solve for its quenched free energy, in the thermodynamic limit and allowing for a first step of replica symmetry breaking.…
Replica-mean-field models have been proposed to decipher the activity of neural networks via a multiply-and-conquer approach. In this approach, one considers limit networks made of infinitely many replicas with the same basic neural…
Symmetry restoration is usually understood as a renormalization group induced phenomenon. In this context, the issue of whether one-loop RG equations can be trusted in predicting symmetry restoration has recently been the subject of much…
We introduce two- and one-dimensional (2D and 1D) systems of two linearly-coupled Gross-Pitaevskii equations (GPEs) with the cubic self-attraction and harmonic-oscillator (HO) trapping potential in each GPE. The system models a…