Related papers: Jamming in multilayer supervised learning models
We study a well known machine learning model -the perceptron- as a simple model of jamming of hard objects. We exhibit two regimes: 1) a convex optimisation regime where jamming is hypostatic and non-critical. 2) a non convex optimisation…
In this Letter, we analyze the quantum dynamics of the perceptron model: a particle is constrained on a $N$-dimensional sphere, with $N\to \infty$, and subjected to a set of randomly placed hard-wall potentials. This model has several…
Recent computer simulations have uncovered the striking difference between the jamming transition of spherical and non-spherical particles. While systems of spherical particles are isostatic at the jamming point, systems of nonspherical…
Random constraint satisfaction problems (CSP) have been studied extensively using statistical physics techniques. They provide a benchmark to study average case scenarios instead of the worst case one. The interplay between statistical…
Gradient descent dynamics in complex energy landscapes, i.e. featuring multiple minima, finds application in many different problems, from soft matter to machine learning. Here, we analyze one of the simplest examples, namely that of soft…
Criticality in statistical physics naturally emerges at isolated points in the phase diagram. Jamming of spheres is not an exception: varying density, it is the critical point that separates the unjammed phase where spheres do not overlap…
Deep neural networks (DNN) with a huge number of adjustable parameters remain largely black boxes. To shed light on the hidden layers of DNN, we study supervised learning by a DNN of width $N$ and depth $L$ consisting of $NL$ perceptrons…
The jamming transition is ubiquitous. It is present in granular matter, colloids, glasses, and many other systems. Yet, it defines a critical point whose properties still need to be fully understood. A major breakthrough came about when the…
Amorphous packings of non-spherical particles such as ellipsoids and spherocylinders are known to be hypostatic: the number of mechanical contacts between particles is smaller than the number of degrees of freedom, thus violating Maxwell's…
The concept of jamming has attracted great research interest due to its broad relevance in soft matter such as liquids, glasses, colloids, foams, and granular materials, and its deep connection to the sphere packing problem and optimization…
The jamming transition of particles with finite-range interactions is characterized by a variety of critical phenomena, including power law distributions of marginal contacts. We numerically study a recently proposed simple model of…
Recurrent neural networks (RNNs) notoriously struggle to learn long-term memories, primarily due to vanishing and exploding gradients. The recent success of state-space models (SSMs), a subclass of RNNs, to overcome such difficulties…
The discontinuous jump in the bulk modulus $B$ at the jamming transition is a consequence of the formation of a critical contact network of spheres that resists compression. We introduce lattice models with underlying under-coordinated…
We present a parallel derivation of the Thouless-Anderson-Palmer (TAP) equations and of an effective potential for the negative perceptron and soft sphere models in high dimension. Both models are continuous constrained satisfaction…
This paper develops a neural jamming phase diagram that interprets the emergence of consciousness in large language models as a critical phenomenon in high-dimensional disordered systems.By establishing analogies with jamming transitions in…
We report an analytical study of the vibrational spectrum of the simplest model of jamming, the soft perceptron. We identify two distinct classes of soft modes. The first kind of modes are related to isostaticity and appear only in the…
We numerically study the jamming transition of frictionless polydisperse spheres in three dimensions. We use an efficient thermalisation algorithm for the equilibrium hard sphere fluid and generate amorphous jammed packings over a range of…
One of the most influential results in neural network theory is the universal approximation theorem [1, 2, 3] which states that continuous functions can be approximated to within arbitrary accuracy by single-hidden-layer feedforward neural…
Jamming criticality defines a universality class that includes systems as diverse as glasses, colloids, foams, amorphous solids, constraint satisfaction problems, neural networks, etc. A particularly interesting feature of this class is…
We outline the general framework of machine learning (ML) methods for multi-scale dynamical modeling of condensed matter systems, and in particular of strongly correlated electron models. Complex spatial temporal behaviors in these systems…