Related papers: How glassy are neural networks?
Sample-to-sample free energy fluctuations in spin-glasses display a markedly different behaviour in finite-dimensional and fully-connected models, namely Gaussian vs. non-Gaussian. Spin-glass models defined on various types of random graphs…
A common theoretical approach to understanding neural networks is to take an infinite-width limit, at which point the outputs become Gaussian process (GP) distributed. This is known as a neural network Gaussian process (NNGP). However, the…
The scaling of fluctuations in the distribution of ground-state energies or costs with the system size N for Ising spin glasses is considered using an extensive set of simulations with the Extremal Optimization heuristic across a range of…
Macroscopic spin ensembles possess brain-like features such as non-linearity, plasticity, stochasticity, selfoscillations, and memory effects, and therefore offer opportunities for neuromorphic computing by spintronics devices. Here we…
We study the Hopfield model with pure $p$-spin interactions with even $p\geq 4$, and a number of patterns, M(N) growing with the system size, $N$, as $M(N) = \a N^{p-1}$. We prove the existence of a critical temperature $\b_p$ characterized…
We consider a system of two-level quantum quasi-spins and gauge bosons put on a 3+1D lattice. As a model of neural network of the brain functions, these spins describe neurons quantum-mechanically, and the gauge bosons describes weights of…
An approach is proposed to the Hopfield model where the mean-field treatment is made for a given set of stored patterns (sample) and then the statistical average over samples is taken. This corresponds to the approach made by Thouless,…
The magnetic systems with disorder form an important class of systems, which are under intensive studies, since they reflect real systems. Such a class of systems is the spin glass one, which combines randomness and frustration. The…
We consider a multi-layer Sherrington-Kirkpatrick spin-glass as a model for deep restricted Boltzmann machines and we solve for its quenched free energy, in the thermodynamic limit and allowing for a first step of replica symmetry breaking.…
The notion of complex energy landscape underpins the intriguing dynamical behaviors in many complex systems ranging from polymers, to brain activity, to social networks and glass transitions. The spin glass state found in dilute magnetic…
We investigate the maximum number of embedded patterns in the two-dimensional Hopfield model. The grand state energies of two specific network states, namely, the energies of the pure-ferromagnetic state and the state of specific one stored…
We study the dynamics of supervised on-line learning of realizable tasks in feed-forward neural networks. We focus on the regime where the number of examples used for training is proportional to the number of input channels N. Using…
In this paper we study the properties of the quenched pressure of a multi-layer spin-glass model (a deep Boltzmann Machine in artificial intelligence jargon) whose pairwise interactions are allowed between spins lying in adjacent layers and…
We devise a deterministic algorithm to efficiently sample high-quality solutions of certain spin-glass systems that encode hard optimization problems. We employ tensor networks to represent the Gibbs distribution of all possible…
At low temperatures the configurational phase space of a macroscopic complex system (e.g., a spin-glass) of $N\sim 10^{23}$ interacting particles may split into an exponential number $\Omega_s \sim \exp({\rm const} \times N)$ of ergodic…
Several recent experiments in biology study systems composed of several interacting elements, for example neuron networks. Normally, measurements describe only the collective behavior of the system, even if in most cases we would like to…
We have analyzed a non-randomly frustrated spin model which exhibits behavior remarkably similar to the phenomenology of structural glasses. The high-temperature disordered phase undergoes a strong first-order transition to a long-range…
Deep neural networks are workhorse models in machine learning with multiple layers of non-linear functions composed in series. Their loss function is highly non-convex, yet empirically even gradient descent minimisation is sufficient to…
We explore a one-to-one correspondence between a neural network (NN) and a statistical mechanical spin model where neurons are mapped to Ising spins and weights to spin-spin couplings. The process of training an NN produces a family of spin…
Glass phases can be stabilized by quenched disorders, as in most spin-glass materials, or self-generated through kinetic freezing in disorder-free systems. A canonical example of the latter is structural glasses, which have been extensively…