Related papers: How glassy are neural networks?
The effects of random magnetic fields are considered in an Ising spin-glass model defined in the limit of infinite-range interactions. The probability distribution for the random magnetic fields is a double Gaussian, which consists of two…
We consider two models of Hopfield-like associative memory with $q$-valued neurons: Potts-glass neural network (PGNN) and parametrical neural network (PNN). In these models neurons can be in more than two different states. The models have…
Given discrete degrees of freedom (spins) on a graph interacting via an energy function, what can be said about the energy local minima and associated inherent structures? Using the lid algorithm in the context of a spin glass energy…
We propose a general framework to extract microscopic interactions from raw configurations with deep neural networks. The approach replaces the modeling Hamiltonian by the neural networks, in which the interaction is encoded. It can be…
We examine learning dynamics in deep recurrent networks, focusing on the behavior near the boundary in the depth-width plane separating under- from over-parametrized networks, known as the interpolation transition. The training data are…
An appropriate model for the random energy landscape in organic glasses is a spatially correlated Gaussian field. We calculated the distribution of the average value of a Gaussian random field in a finite domain. The results of the…
We propose and study a model with glassy behavior. The state space of the model is given by all triangulations of a sphere with $n$ nodes, half of which are red and half are blue. Red nodes want to have 5 neighbors while blue ones want 7.…
We investigate the energy landscape of two dimensional network models for covalent glasses by means of the lid algorithm. For three different particle densities and for a range of network sizes, we exhaustively analyse many configuration…
Bifurcation theory is a powerful tool for studying how the dynamics of a neural network model depends on its underlying neurophysiological parameters. However, bifurcation theory has been developed mostly for smooth dynamical systems and…
We introduce Graph Hopfield Networks, whose energy function couples associative memory retrieval with graph Laplacian smoothing for node classification. Gradient descent on this joint energy yields an iterative update interleaving Hopfield…
We study two dimensional frustrated but non-disordered systems applying a replica approach to a stripe forming model with competing interactions. The phenomenology of the model is representative of several well known systems, like high-Tc…
Obtaining the low-energy configurations of spin glasses that have rugged energy landscapes is of direct relevance to combinatorial optimization and fundamental science. Search-based heuristics have difficulty with this task due to the…
Dense Associative Memories or modern Hopfield networks permit storage and reliable retrieval of an exponentially large (in the dimension of feature space) number of memories. At the same time, their naive implementation is non-biological,…
In the past decade, significant strides in deep learning have led to numerous groundbreaking applications. Despite these advancements, the understanding of the high generalizability of deep learning, especially in such an over-parametrized…
The dynamics of an extremely diluted neural network with high order synapses acting as corrections to the Hopfield model is investigated. As in the fully connected case, the high order terms may strongly improve the storage capacity of the…
We propose stochastic, non-parametric activation functions that are fully learnable and individual to each neuron. Complexity and the risk of overfitting are controlled by placing a Gaussian process prior over these functions. The result is…
We study the behavior of droplets for two dimensional Ising spin glasses with Gaussian interactions. We use an exact matching algorithm which enables study of systems with linear dimension L up to 240, which is larger than is possible with…
We study a two-dimensional compressible Ising spin glass at constant volume. The spin interactions are coupled to the distance between neighboring particles in the Edwards-Anderson model with +/- J interactions. We find that the energy of a…
The core idea of stochastic stability is that thermodynamic observables must be robust under small (random) perturbations of the quenched Gibbs measure. Combining this idea with the cavity field technique, which aims to measure the free…
It is often stated that if one is presented with a snapshot of the positions of the molecules of a glass and one of a liquid, one is unable to tell the difference. Here we argue instead that given several such snapshots taken over a…