Related papers: Geometric Entropy and Retrieval Phase Transitions …
Our work combines aspects of three promising paradigms in machine learning, namely, attention mechanism, energy-based models, and associative memory. Attention is the power-house driving modern deep learning successes, but it lacks clear…
We measured the energy efficiency of information erasure using silicon DRAM cells capable of counting charges on capacitors at the single-electron level. Our measurements revealed that the efficiency decreased as the erasure error…
The Hopfield model is a paradigmatic model of neural networks that has been analyzed for many decades in the statistical physics, neuroscience, and machine learning communities. Inspired by the manifold hypothesis in machine learning, we…
The large spatial/frequency scale of hyperspectral and airborne magnetic and gravitational data causes memory issues when using convolutional neural networks for (sub-) surface characterization. Recently developed fully reversible networks…
We present results for two different kinds of high order connections between neurons acting as corrections to the Hopfield model. Equilibrium properties are analyzed using the replica mean-field theory and compared with numerical…
We study a simple extended model of oscillator neural networks capable of storing sparsely coded phase patterns, in which information is encoded both in the mean firing rate and in the timing of spikes. Applying the methods of statistical…
Current LLM agents lack principled mechanisms for managing persistent memory across long interaction horizons. We present a biologically-grounded memory architecture comprising six cognitive mechanisms: (1) sleep-phase consolidation, (2)…
In \cite{Hop82}, Hopfield introduced a \emph{Hebbian} learning rule based neural network model and suggested how it can efficiently operate as an associative memory. Studying random binary patterns, he also uncovered that, if a small…
Erasure of the binary memory, 0 or 1, is an essential step for digital computation involving irreversible logic operations. The erasure of a bit of a classical bit of memory is accompanied by the evolution of a minimum amount of heat set by…
Knowledge graphs used for retrieval treat all facts as equally current. Existing temporal approaches apply uniform decay, using a single forgetting curve regardless of knowledge type. We show this is fundamentally misspecified: different…
We show how a Hopfield network with modifiable recurrent connections undergoing slow Hebbian learning can extract the underlying geometry of an input space. First, we use a slow/fast analysis to derive an averaged system whose dynamics…
Associative memory, traditionally modeled by Hopfield networks, enables the retrieval of previously stored patterns from partial or noisy cues. Yet, the local computational principles which are required to enable this function remain…
Various Graph Neural Networks (GNNs) have been successful in analyzing data in non-Euclidean spaces, however, they have limitations such as oversmoothing, i.e., information becomes excessively averaged as the number of hidden layers…
A statistical mechanical distance constraint model (DCM) is presented that explicitly accounts for network rigidity among constraints present within a system. Constraints are characterized by local microscopic free energy functions.…
We numerically study thermodynamic and structural properties of the one-component Gaussian core model (GCM) at very high densities. The solid-fluid phase boundary is carefully determined. We find that the density dependence of both the…
Holographic studies of the entanglement entropy of field theories dual to charged and neutral black holes in asymptotically global AdS4 spacetimes are presented. The goal is to elucidate various properties of the quantity that are peculiar…
One of the most influential results in neural network theory is the universal approximation theorem [1, 2, 3] which states that continuous functions can be approximated to within arbitrary accuracy by single-hidden-layer feedforward neural…
The brain may adopt the strategy of lateral predictive coding (LPC) to construct optimal internal representations for salient features in input sensory signals, reducing the energetic cost of information transmission. Here we first consider…
In systems undergoing second order phase transitions, the temperature integral of the specific heat over temperature from zero to the critical temperature is the same in both the normal and ordered phases. This entropy balance relates the…
We demonstrate the accurate calculation of entropies and free energies for a variety of liquid metals using an extension of the two phase thermodynamic (2PT) model based on a decomposition of the velocity autocorrelation function into…