Related papers: Disordered Dynamics in High Dimensions: Connection…
Dynamical mean field theory (DMFT) is a tool that allows to analyze the stochastic dynamics of $N$ interacting degrees of freedom in terms of a self-consistent $1$-body problem. In this work, focusing on models of ecosystems, we present the…
Modern machine learning models are typically trained via multi-pass stochastic gradient descent (SGD) with small batch sizes, and understanding their dynamics in high dimensions is of great interest. However, an analytical framework for…
Diagonal linear networks (DLNs) are a tractable model that captures several nontrivial behaviors in neural network training, such as initialization-dependent solutions and incremental learning. These phenomena are typically studied in…
Dynamical Mean-Field Theory (DMFT) is a powerful theoretical framework for analyzing systems with many interacting degrees of freedom. This tutorial provides an accessible introduction to DMFT. We begin with a linear model where the DMFT…
Understanding the inductive bias and generalization properties of large overparametrized machine learning models requires to characterize the dynamics of the training algorithm. We study the learning dynamics of large two-layer neural…
We perform an average case analysis of the generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant "high-dimensional" regime where the number of free parameters in the network is on…
Understanding the learning dynamics of neural networks is one of the key issues for the improvement of optimization algorithms as well as for the theoretical comprehension of why deep neural nets work so well today. In this paper, we…
Dynamical Mean Field Theory (DMFT) provides an asymptotic description of the dynamics of macroscopic observables in certain disordered systems. Originally pioneered in the context of spin glasses by Sompolinsky and Zippelius (1982), it has…
Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data and rely on overparameterized models, where classical low-dimensional intuitions break down. In particular, the proportional regime where…
The study of nonequilibrium phenomena in correlated lattice systems has developed into an active and exciting branch of condensed matter physics. This research field provides rich new insights that could not be obtained from the study of…
We study a class of deterministic flows in ${\mathbb R}^{d\times k}$, parametrized by a random matrix ${\boldsymbol X}\in {\mathbb R}^{n\times d}$ with i.i.d. centered subgaussian entries. We characterize the asymptotic behavior of these…
This paper proposes a novel parametric identification approach for linear systems using Deep Learning (DL) and the Modified Relay Feedback Test (MRFT). The proposed methodology utilizes MRFT to reveal distinguishing frequencies about an…
Dynamical Mean-Field Theory (DMFT) has opened new perspectives for the investigation of strongly correlated electron systems and greatly improved our understanding of correlation effects in models and materials. In contrast to…
We study a fully connected Hopfield-type associative memory network with online activity-dependent synaptic plasticity, where neural states and synaptic couplings coevolve during retrieval. Using the generating-functional formalism, we…
It is described how one comes to the Wigner-Dyson random matrix theory (RMT) starting from a model of a disordered metal. The lectures start with a historical introduction where basic ideas of the RMT and theory of disordered metals are…
Mean-field theories have proven to be efficient tools for exploring diverse phases of matter, complementing alternative methods that are more precise but also more computationally demanding. Conventional mean-field theories often fall short…
Large nonlinear recurrent neural networks with random couplings generate high-dimensional, potentially chaotic activity whose structure is of interest in neuroscience and other fields. A fundamental object encoding the collective structure…
Dynamical density functional theory (DDFT) is a powerful variational framework to study the nonequilibrium properties of colloids by only considering a time-dependent one-body number density. Despite the large number of recent successes,…
A density matrix describes the statistical state of a quantum system. It is a powerful formalism to represent both the quantum and classical uncertainty of quantum systems and to express different statistical operations such as measurement,…
We solve the nonequilibrium dynamical mean-field theory (DMFT) using matrix product states (MPS). This allows us to treat much larger bath sizes and by that reach substantially longer times (factor $\sim$ 2 -- 3) than with exact…