Related papers: The high-dimensional asymptotics of first order me…
Modern machine learning models are typically trained via multi-pass stochastic gradient descent (SGD) with small batch sizes, and understanding their dynamics in high dimensions is of great interest. However, an analytical framework for…
We establish non-asymptotic bounds on the finite-sample behavior of generalized first-order iterative algorithms -- including gradient-based optimization methods and approximate message passing (AMP) -- with Gaussian data matrices and…
Dynamical Mean Field Theory (DMFT) provides an asymptotic description of the dynamics of macroscopic observables in certain disordered systems. Originally pioneered in the context of spin glasses by Sompolinsky and Zippelius (1982), it has…
We provide an overview of high dimensional dynamical systems driven by random matrices, focusing on applications to simple models of learning and generalization in machine learning theory. Using both cavity method arguments and path…
The probability distributions, as well as the mean values of stochastic currents and fluxes, associated with a driven Langevin process, provide a good and topologically protected measure of how far a stochastic system is driven out of…
This paper investigates asymptotic behaviors of gradient descent algorithms (particularly accelerated gradient descent and stochastic gradient descent) in the context of stochastic optimization arising in statistics and machine learning…
We prove that all 'gradient span algorithms' have asymptotically deterministic behavior on scaled Gaussian random functions as the dimension tends to infinity. In particular, this result explains the counterintuitive phenomenon that…
The solutions to surface evolution problems like mean curvature flow can be expressed as value functions of suitable stochastic control problems, obtained as limit of a family of regularised control problems. The control-theoretical…
We study the learning dynamics of a multi-pass, mini-batch Stochastic Gradient Descent (SGD) procedure for empirical risk minimization in high-dimensional multi-index models with isotropic random data. In an asymptotic regime where the…
In this paper, we study inference for high-dimensional data characterized by small sample sizes relative to the dimension of the data. In particular, we provide an infinite-dimensional framework to study statistical models that involve…
Assignment flows denote a class of dynamical models for contextual data labeling (classification) on graphs. We derive a novel parametrization of assignment flows that reveals how the underlying information geometry induces two processes…
We study the time-averaged flow in a model of particles that randomly hop on a finite directed graph. In the limit as the number of particles and the time window go to infinity but the graph remains finite, the large-deviation rate…
We study the short-time asymptotical behavior of stochastic flows on \mathbb{R} in the \sup-norm. The results are stated in terms of a Gaussian process associated with the covariation of the flow. In case the Gaussian process has a…
This work addresses the problem of learning the dynamics of high-dimensional probability densities over time using unlabeled samples, without assuming access to trajectory information. We introduce two-parameter flows that learn only…
We extend a general result showing that the asymptotic behavior of high moments, factorial or standard, of random variables, determines the asymptotically normality, from the one dimensional to the multidimensional setting. This approach…
Approximate Message Passing (AMP) algorithms are a class of iterative procedures for computationally-efficient estimation in high-dimensional inference and estimation tasks. Due to the presence of an 'Onsager' correction term in its…
In this paper we study the large time asymptotics of the flow of a dynamical system $X'=b(X)$ posed in the $d$-dimensional torus. Rather than using the classical unique ergodicity condition which is not fulfilled if $b$ vanishes at…
A successful method to describe the asymptotic behavior of various deterministic and stochastic processes such as asymptotically autonomous differential equations or stochastic approximation processes is to relate it to an appropriately…
We study asymptotic behaviour of stochastic approximation procedures with three main characteristics: truncations with random moving bounds, a matrix valued random step-size sequence, and a dynamically changing random regression function.…
Time-irreversible stochastic processes are frequently used in natural sciences to explain non-equilibrium phenomena and to design efficient stochastic algorithms. Our main goal in this thesis is to analyse their dynamics by means of large…