Related papers: Beyond storage capacity in a single model neuron: …
We investigate the structure of Parisi measures, the functional order parameters of mixed p-spin models in mean field spin glasses. In the absence of external field, we prove that a Parisi measure satisfies the following properties. First,…
We discuss mean field theory of glasses without quenched disorder focusing on the justification of the replica approach to thermodynamics. We emphasize the assumptions implicit in this method and discuss how they can be verified. The…
A central challenge in machine learning is to distinguish genuine structure from chance correlations in high-dimensional data. In this work, we address this issue for the perceptron, a foundational model of neural computation. Specifically,…
Using Monte Carlo simulations, we study in detail the overlap distribution for individual samples for several spin-glass models including the infinite-range Sherrington-Kirkpatrick model, short-range Edwards-Anderson models in three and…
We study random constraint satisfaction problems (CSPs) in the unsatisfiable regime. We relate the structure of near-optimal solutions for any Max-CSP to that for an associated spin glass on the hypercube, using the Guerra-Toninelli…
Methods for understanding classical disordered spin systems with interactions conforming to some idealized graphical structure are well developed. The equilibrium properties of the Sherrington-Kirkpatrick model, which has a densely…
We analyze the replica-symmetry-breaking construction in the Sherrington-Kirkpatrick model of a spin glass. We present a general scheme for deriving an exact asymptotic behavior near the critical temperature of the solution with an…
Multi-electrode arrays covering several square millimeters of neural tissue provide simultaneous access to population signals such as extracellular potentials and spiking activity of one hundred or more individual neurons. The…
We study a one-dimensional version of the Hopfield model with long, but finite range interactions below the critical temperature. In the thermodynamic limit we obtain large deviation estimates for the distribution of the ``local'' overlaps,…
Standard large deviation estimates or the use of the Hubbard-Stratonovich transformation reduce the analysis of the distribution of the overlap parameters essentially to that of an explicitly known random function $\Phi_{N,\b}$ on $\R^M$.…
While the Gibbs states of spin glass models have been noted to have an erratic dependence on temperature, one may expect the mean over the disorder to produce a continuously varying ``quenched state''. The assumption of such continuity in…
We study the role played by the dilution in the average behavior of a perceptron model with continuous coupling with the replica method. We analyze the stability of the replica symmetric solution as a function of the dilution field for the…
We develop a statistical mechanical approach based on the replica method to study the design space of deep and wide neural networks constrained to meet a large number of training data. Specifically, we analyze the configuration space of the…
We continue the work of a series of previous studies of a mathematical model that describes the mean-field limit behavior of a homogeneous network of excitatory point spiking neurons. Contrary to other models, here noise is intrinsic to the…
Neural networks achieve remarkable performance through superposition: encoding multiple features as overlapping directions in activation space rather than dedicating individual neurons to each feature. This challenges interpretability, yet…
We study the dynamics of a particle in a one-dimensional Gaussian random potential with logarithmic correlations. It was shown in previous studies that the model exhibits a dynamical transition between two subdiffusive phases. We…
For a very general class of probability distributions in disordered Ising spin systems, in the thermodynamical limit, we prove the following property for overlaps among real replicas. Consider the overlaps among s replicas. Add one replica…
An obstacle to artificial general intelligence is set by continual learning of multiple tasks of different nature. Recently, various heuristic tricks, both from machine learning and from neuroscience angles, were proposed, but they lack a…
We study self-programming in recurrent neural networks where both neurons (the `processors') and synaptic interactions (`the programme') evolve in time simultaneously, according to specific coupled stochastic equations. The interactions are…
Square Wave Perceptrons (SWPs) form a class of neural network models with oscillating activation function that exhibit intriguing ``hardness'' properties in the high-dimensional limit at a fixed constraint density $\alpha = O(1)$. In this…