Related papers: Jamming in multilayer supervised learning models

When performing data classification over a stream of continuously occurring instances, a key challenge is to develop an open-world classifier that anticipates instances from an unknown class. Studies addressing this problem, typically…

Recurrent neural networks have gained widespread use in modeling sequential data. Learning long-term dependencies using these models remains difficult though, due to exploding or vanishing gradients. In this paper, we draw connections…

Deep learning models have proven enormously successful at using multiple layers of representation to learn relevant features of structured data. Encoding physical symmetries into these models can improve performance on difficult tasks, and…

We study the glass and jamming transition of finite-dimensional models of simple liquids: hard- spheres, harmonic spheres and more generally bounded pair potentials that modelize frictionless spheres in interaction. At finite temperature,…

Oversmoothing is a common phenomenon in a wide range of Graph Neural Networks (GNNs) and Transformers, where performance worsens as the number of layers increases. Instead of characterizing oversmoothing from the view of complete collapse…

The Spiral Model (SM) corresponds to a new class of kinetically constrained models introduced in joint works with D.S. Fisher [8,9]. They provide the first example of finite dimensional models with an ideal glass-jamming transition. This is…

Continuum mechanics simulators, numerically solving one or more partial differential equations, are essential tools in many areas of science and engineering, but their performance often limits application in practice. Recent modern machine…

Tensor models play an increasingly prominent role in many fields, notably in machine learning. In several applications, such as community detection, topic modeling and Gaussian mixture learning, one must estimate a low-rank signal from a…

Rigidity transitions induced by the formation of system-spanning disordered rigid clusters, like the jamming transition, can be well-described in most physically relevant dimensions by mean-field theories. A dynamical mean-field theory…

Here we investigate the single-layer linearized perceptron near the SAT-UNSAT transition point as a prototypical model of the convex continuous satisfaction problems. The simplicity of the model allows us to take into account the effects of…

In this work, we present and study Continuous Generative Neural Networks (CGNNs), namely, generative models in the continuous setting: the output of a CGNN belongs to an infinite-dimensional function space. The architecture is inspired by…

Deep learning has been immensely successful at a variety of tasks, ranging from classification to AI. Learning corresponds to fitting training data, which is implemented by descending a very high-dimensional loss function. Understanding…

Neural network based architectures used for sound recognition are usually adapted from other application domains such as image recognition, which may not harness the time-frequency representation of a signal. The ConditionaL Neural Networks…

The recently proposed Multi-Layer Convolutional Sparse Coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new interpretation of Convolutional Neural Networks (CNNs). Under this framework, the…

We study numerically a system of athermal, overdamped, frictionless spheres, as in a non-Brownian suspension, in two and three dimensions. Compressing the system isotropically at a fixed rate $\dot\epsilon$, we investigate the critical…

A class of shape-invariant bound-state problems which represent transition in a two-level system introduced earlier are generalized to include arbitrary energy splittings between the two levels as well as intensity-dependent interactions.…

Monotonic neural networks have recently been proposed as a way to define invertible transformations. These transformations can be combined into powerful autoregressive flows that have been shown to be universal approximators of continuous…

Hyperuniformity characterizes a state of matter that is poised at a critical point at which density or volume-fraction fluctuations are anomalously suppressed at infinite wavelengths. Recently, much attention has been given to the link…

Multi-layer perceptrons (MLPs) are a standard tool for learning and function approximation, but they inherently yield outputs that are globally smooth. As a result, they struggle to represent functions that are continuous yet deliberately…

A renormalization scheme is developed to study an anisotropic quantum XY spin chain in a quasiperiodic transverse field. The critical phase of the quasi-particle excitations of the model with fractal wave functions exists in a finite…