Related papers: Jamming in multilayer supervised learning models

The canonical tensor model (CTM) is a tensor model proposing a classically and quantum mechanically consistent model of gravity, formulated as a first-class constraint system with structural similarities to the ADM formalism of general…

Understanding the fundamental principles behind the massive success of neural networks is one of the most important open questions in deep learning. However, due to the highly complex nature of the problem, progress has been relatively…

We study the machine learning task for models with operators mapping between the Wasserstein space of probability measures and a space of functions, like e.g. in mean-field games/control problems. Two classes of neural networks, based on…

Scientific machine learning (SciML) increasingly requires models that capture multimodal conditional uncertainty arising from ill-posed inverse problems, multistability, and chaotic dynamics. While recent work has favored highly expressive…

Machine learning has become a premier tool in physics and other fields of science. It has been shown that the quantum mechanical scattering problem can not only be solved with such techniques, but it was argued that the underlying neural…

We use machine learning to classify examples of braids (or flat braids) as trivial or non-trivial. Our ML takes form of supervised learning using neural networks (multilayer perceptrons). When they achieve good results in classification, we…

Soft, disordered, micro-structured materials are ubiquitous in nature and industry, and are different from ordinary fluids or solids, with unusual, interesting static and flow properties. The transition from fluid to solid -at the so-called…

Modern large language models (LLMs) excel at tasks that require storing and retrieving knowledge, such as factual recall and question answering. Transformers are central to this capability because they can encode information during training…

We numerically produce fully amorphous assemblies of frictionless spheres in three dimensions and study the jamming transition these packings undergo at large volume fractions. We specify four protocols yielding a critical value for the…

In this work we provide an overview of jamming transitions in two dimensional systems focusing on the limit of frictionless particle interactions in the absence of thermal fluctuations. We first discuss jamming in systems with short range…

In this paper, we study the feature learning ability of two-layer neural networks in the mean-field regime through the lens of kernel methods. To focus on the dynamics of the kernel induced by the first layer, we utilize a two-timescale…

Athermal packings of soft repulsive spheres exhibit a sharp jamming transition in the thermodynamic limit. Upon further compression, various structural and mechanical properties display clean power-law behavior over many decades in…

In applications ranging from communications to genetics, signals can be modeled as lying in a union of subspaces. Under this model, signal coefficients that lie in certain subspaces are active or inactive together. The potential subspaces…

Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and Memory Networks which contain memory are popularly used to learn patterns in sequential data. Sequential data has long sequences that hold relationships. RNN can…

In the mean field regime, neural networks are appropriately scaled so that as the width tends to infinity, the learning dynamics tends to a nonlinear and nontrivial dynamical limit, known as the mean field limit. This lends a way to study…

In this paper, we consider parameter recovery for non-overlapping convolutional neural networks (CNNs) with multiple kernels. We show that when the inputs follow Gaussian distribution and the sample size is sufficiently large, the squared…

Multilingual language models (MLLMs) have demonstrated remarkable abilities to transfer knowledge across languages, despite being trained without explicit cross-lingual supervision. We analyze the parameter spaces of three MLLMs to study…

Resonances, also known as quasinormal modes (QNM) in the non-Hermitian case, play a ubiquitous role in all domains of physics ruled by wave phenomena, notably in continuum mechanics, acoustics, electrodynamics, and quantum theory. The…

We analytically derive the geometrical structure of the weight space in multilayer neural networks (MLN), in terms of the volumes of couplings associated to the internal representations of the training set. Focusing on the parity and…

Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, and many other domains. The involved deep neural network architectures and computational issues have been well studied in machine…