相关论文: Replica symmetry breaking in dense neural networks

The phenomenon of replica symmetry breaking is investigated for the retrieval phases of Hopfield-type network models. The basic calculation is done for the generalized version of the standard model introduced by Horner [1] and by…

Statistical mechanics of spin glasses is one of the main strands toward a comprehension of information processing by neural networks and learning machines. Tackling this approach, at the fairly standard replica symmetric level of…

This work presents a statistical mechanics characterization of neural networks, motivated by the replica symmetry breaking (RSB) phenomenon in spin glasses. A Hopfield-type spin glass model is constructed from a given feedforward neural…

Some interesting recent advances in the theoretical understanding of neural networks have been informed by results from the physics of disordered many-body systems. Motivated by these findings, this work uses the replica technique to study…

In this paper we adapt the broken replica interpolation technique (developed by Francesco Guerra to deal with the Sherrington-Kirkpatrick model, namely a pairwise mean-field spin-glass whose couplings are i.i.d. standard Gaussian variables)…

In this paper we continue our investigation on the high storage regime of a neural network with Gaussian patterns. Through an exact mapping between its partition function and one of a bipartite spin glass (whose parties consist of Ising and…

During the last years, through the combined effort of the insight, coming from physical intuition and computer simulation, and the exploitation of rigorous mathematical methods, the main features of the mean field Sherrington-Kirkpatrick…

Glassiness occurs when disorder and frustration cause local degrees of freedom to freeze despite the lack of long-range order. In systems of interacting bosons, such glassiness may involve a purely quantum degree of…

In this work we present a rigorous and straightforward method to detect the onset of the instability of replica-symmetric theories in information processing systems, which does not require a full replica analysis as in the method originally…

We discuss replica symmetry breaking (RSB) in spin glasses. We update work in this area, from both the analytical and numerical points of view. We give particular attention to the difficulties stressed by Newman and Stein concerning the…

Recently a daily routine for associative neural networks has been proposed: the network Hebbian-learns during the awake state (thus behaving as a standard Hopfield model), then, during its sleep state, optimizing information storage, it…

Replica symmetry breaking (RSB) underlies the complex organization of disordered systems, yet quantitative validation beyond $N \sim 100$ spins has remained computationally challenging. We use quantum annealing to access ground states of…

We consider a multi-layer Sherrington-Kirkpatrick spin-glass as a model for deep restricted Boltzmann machines and we solve for its quenched free energy, in the thermodynamic limit and allowing for a first step of replica symmetry breaking.…

We perform the replica symmetry breaking (RSB) in the vicinity of the point of instability of the replica symmetric solution in the model of axial quadrupolar glass. It is shown that the solution with the first stage RSB is stable against…

The standard Hopfield model for associative neural networks accounts for biological Hebbian learning and acts as the harmonic oscillator for pattern recognition, however its maximal storage capacity is $\alpha \sim 0.14$, far from the…

Spin glasses are quintessential examples of complex matter. Although much about their order remains uncertain, abstract models of them inform, e.g., the classification of combinatorial optimization problems, the magnetic ordering in metals…

We develop a statistical mechanical approach based on the replica method to study the design space of deep and wide neural networks constrained to meet a large number of training data. Specifically, we analyze the configuration space of the…

We consider dense, associative neural-networks trained by a teacher (i.e., with supervision) and we investigate their computational capabilities analytically, via statistical-mechanics of spin glasses, and numerically, via Monte Carlo…

We derive, within the replica formalism, a generalisation of the Crisanti-Sommers formula to describe the large deviation function (LDF) ${\cal L}(e)$ for the speed-$N$ atypical fluctuations of the intensive ground-state energy $e$ of a…

Restricted Boltzmann Machines are key tools in Machine Learning and are described by the energy function of bipartite spin-glasses. From a statistical mechanical perspective, they share the same Gibbs measure of Hopfield networks for…