Related papers: Disordered Dynamics in High Dimensions: Connection…
In this work, we analyze various scaling limits of the training dynamics of transformer models in the feature learning regime. We identify the set of parameterizations that admit well-defined infinite width and depth limits, allowing the…
Using dynamical mean-field theory (DMFT) we study a simplified model for heterostructures involving superconductors. The system is driven out-of-equilibrium by a voltage bias, imposed as an imbalance of chemical potential at the interface.…
We present a systematic and reliable methodology, termed hierarchical mean-field theory (HMFT), to study and predict the behavior of strongly coupled many-particle systems. HMFT is a simple approximation, based upon group theoretical…
In this work, we present some applications of random matrix theory for the training of deep neural networks. Recently, random matrix theory (RMT) has been applied to the overfitting problem in deep learning. Specifically, it has been shown…
Recent evidence has shown the existence of a so-called double-descent and even triple-descent behavior for the generalization error of deep-learning models. This important phenomenon commonly appears in implemented neural network…
Datasets such as images, text, or movies are embedded in high-dimensional spaces. However, in important cases such as images of objects, the statistical structure in the data constrains samples to a manifold of dramatically lower…
This article conducts a large dimensional study of a simple yet quite versatile classification model, encompassing at once multi-task and semi-supervised learning, and taking into account uncertain labeling. Using tools from random matrix…
State space models (SSMs) have shown remarkable empirical performance on many long sequence modeling tasks, but a theoretical understanding of these models is still lacking. In this work, we study the learning dynamics of linear SSMs to…
We present an inhomogeneous dynamical mean field theory (I-DMFT) that is suitable to investigate electron-lattice interactions in non-translationally invariant and/or inhomogeneous systems. The presented approach, whose only assumption is…
Quantum computers (QC) could harbor the potential to significantly advance materials simulations, particularly at the atomistic scale involving strongly correlated fermionic systems where an accurate description of quantum many-body effects…
Many consequential real-world systems, like wind fields and ocean currents, are dynamic and hard to model. Learning their governing dynamics remains a central challenge in scientific machine learning. Dynamic Mode Decomposition (DMD)…
Since the first investigation of the Hubbard model in the limit of infinite dimensions by Metzner and Vollhardt, dynamical mean-field theory (DMFT) has become a very powerful tool for the investigation of lattice models of correlated…
Multiway datasets are commonly analyzed using unsupervised matrix and tensor factorization methods to reveal underlying patterns. Frequently, such datasets include timestamps and could correspond to, for example, health-related measurements…
We extend the nonequilibrium dynamical mean field (DMFT) formalism to inhomogeneous systems by adapting the "real-space" DMFT method to Keldysh Green's functions. Solving the coupled impurity problems using strong-coupling perturbation…
We study the high-dimensional training dynamics of a shallow neural network with quadratic activation in a teacher-student setup. We focus on the extensive-width regime, where the teacher and student network widths scale proportionally with…
Learning dynamical models from data plays a vital role in engineering design, optimization, and predictions. Building models describing dynamics of complex processes (e.g., weather dynamics, or reactive flows) using empirical knowledge or…
Dynamical Systems (DS) are an effective and powerful means of shaping high-level policies for robotics control. They provide robust and reactive control while ensuring the stability of the driving vector field. The increasing complexity of…
Deep learning systems achieve remarkable empirical performance, yet the stability of the training process itself remains poorly understood. Training unfolds as a high-dimensional dynamical system in which small perturbations to…
Magnetic particle imaging reconstructs tracer distributions using a system matrix obtained through time-consuming, noise-prone calibration measurements. Methods for addressing imperfections in measured system matrices increasingly rely on…
Stochastic dynamical systems are ubiquitous in physics, biology, and engineering, where both deterministic drifts and random fluctuations govern system behavior. Learning these dynamics from data is particularly challenging in…