Related papers: Geometric Entropy and Retrieval Phase Transitions …
The Hopfield network (HN) is a classical model of associative memory whose dynamics are closely related to the Ising spin system with 2-body interactions. Stored patterns are encoded as minima of an energy function shaped by a Hebbian…
Spin-glass models of associative memories are a cornerstone between statistical physics and theoretical neuroscience. In these networks, stochastic spin-like units interact through a synaptic matrix shaped by local Hebbian learning. In…
Topological phase transitions are found in a variety of systems and were shown to be deeply related with a thermodynamic description through scaling relations. Here, we investigate the entanglement entropy, which is a quantity that captures…
Kernel-based learning methods such as Kernel Logistic Regression (KLR) can substantially increase the storage capacity of Hopfield networks, but the principles governing their performance and stability remain largely uncharacterized. This…
Continual learning systems operating in fixed-dimensional spaces face a fundamental geometric barrier: the flat manifold problem. When experience is represented as a linear trajectory in Euclidean space, the geodesic distance between…
We study a class of Hopfield models where the memories are represented by a mixture of Gaussian and binary variables and the neurons are Ising spins. We study the properties of this family of models as the relative weight of the two kinds…
Recently a daily routine for associative neural networks has been proposed: the network Hebbian-learns during the awake state (thus behaving as a standard Hopfield model), then, during its sleep state, optimizing information storage, it…
Computation fundamentally separates time from space: nondeterministic search is exponential in time but polynomially simulable in space (Savitch's Theorem). We propose that the brain physically instantiates a biological variant of this…
We reconstruct entanglement thermodynamics by means of Hessian geometry, since this method exactly generalizes thermodynamics into much wider exponential family cases including quantum entanglement. Starting with the correct first law of…
We studied the computational properties of an attractor neural network (ANN) with different network topologies. Though fully connected neural networks exhibit, in general, a good performance, they are biologically unrealistic, as it is…
Brain-inspired computing aims to mimic cognitive functions like associative memory, the ability to recall complete patterns from partial cues. Memristor technology offers promising hardware for such neuromorphic systems due to its potential…
The classic Landauer bound can be lowered when erasure errors are permitted. Here we point out that continuous phase transitions characterized by an order parameter can also be viewed as information erasure by resetting a certain number of…
Near horizons, quantum fields of low spin exhibit densities of states that behave asymptotically like 1+1 dimensional conformal field theories. In effective field theory, imposing some short-distance cutoff, one can compute thermodynamic…
The Hopfield associative memory model stores random patterns in synaptic couplings according to Hebb's rule and retrieves them through gradient descent on an energy function. This conventional setting, where neurons are assumed to have…
The Hopfield recurrent neural network is a classical auto-associative model of memory, in which collections of symmetrically-coupled McCulloch-Pitts neurons interact to perform emergent computation. Although previous researchers have…
In "Large Associative Memory Problem in Neurobiology and Machine Learning," Dmitry Krotov and John Hopfield introduced a general technique for the systematic construction of neural ordinary differential equations with non-increasing energy…
We study extremely diluted spin models of neural networks in which the connectivity evolves in time, although adiabatically slowly compared to the neurons, according to stochastic equations which on average aim to reduce frustration. The…
Models of interacting complex systems provide the fundamental statistical physics reference frame for the study and the understanding of associative memories, machine learning, and the dynamics of neural networks. On the other hand,…
We present a nonparametric interpretation for deep learning compatible modern Hopfield models and utilize this new perspective to debut efficient variants. Our key contribution stems from interpreting the memory storage and retrieval…
We develop a supervised machine learning algorithm that is able to learn topological phases of finite condensed matter systems from bulk data in real lattice space. The algorithm employs diagonalization in real space together with any…