Related papers: A Dynamical Central Limit Theorem for Shallow Neur…

The mean field (MF) theory of multilayer neural networks centers around a particular infinite-width scaling, where the learning dynamics is closely tracked by the MF limit. A random fluctuation around this infinite-width limit is expected…

Understanding deep neural networks (DNNs) is a key challenge in the theory of machine learning, with potential applications to the many fields where DNNs have been successfully used. This article presents a scaling limit for a DNN being…

On a variety of tasks, the performance of neural networks predictably improves with training time, dataset size and model size across many orders of magnitude. This phenomenon is known as a neural scaling law. Of fundamental importance is…

We prove two theorems related to the Central Limit Theorem (CLT) for Martin-L\"of Random (MLR) sequences. Martin-L\"of randomness attempts to capture what it means for a sequence of bits to be "truly random". By contrast, CLTs do not make…

Although overparameterized models have achieved remarkable practical success, their theoretical properties, particularly their generalization behavior, remain incompletely understood. The well known double descents phenomenon suggests that…

Dropout is a standard training technique for neural networks that consists of randomly deactivating units at each step of their gradient-based training. It is known to improve performance in many settings, including in the large-scale…

Suppose $B_i:= B(p,r_i)$ are nested balls of radius $r_i$ about a point $p$ in a dynamical system $(T,X,\mu)$. The question of whether $T^i x\in B_i$ infinitely often (i. o.) for $\mu$ a.e.\ $x$ is often called the shrinking target problem.…

We study the high-dimensional training dynamics of a shallow neural network with quadratic activation in a teacher-student setup. We focus on the extensive-width regime, where the teacher and student network widths scale proportionally with…

Dynamical systems with $\epsilon$ small random perturbations appear in both continuous mechanical motions and discrete stochastic chemical kinetics. The present work provides a detailed analysis of the central limit theorem (CLT), with a…

This paper studies the fluctuations of the signal-to-noise ratio (SNR) of minimum variance distorsionless response (MVDR) filters implementing diagonal loading in the estimation of the covariance matrix. Previous results in the signal…

We discuss the spectral asymptotics of some open subsets of the real line with random fractal boundary and of a random fractal, the continuum random tree. In the case of open subsets with random fractal boundary we establish the existence…

We perform an average case analysis of the generalization dynamics of large neural networks trained using gradient descent. We study the practically-relevant "high-dimensional" regime where the number of free parameters in the network is on…

We consider asymptotic behavior of Fourier transforms of stationary ergodic sequences with finite second moments. We establish a central limit theorem (CLT) for almost all frequencies and also an annealed CLT. The theorems hold for all…

We consider linear two-time-scale stochastic approximation algorithms driven by martingale noise. Recent applications in machine learning motivate the need to understand finite-time error rates, but conventional stochastic approximation…

In this article, we revisit the question of fluctuations of linear statistics of beta ensembles in the single cut and non-critical regime for general potentials $V$ under mild regularity and growth assumptions. Our main objective is to…

We establish limit theorems involving weak convergence of multiple generations of critical and supercritical branching processes. These results arise naturally when dealing with the joint asymptotic behavior of functionals defined in terms…

This work develops a mean-field analysis for the asymptotic behavior of deep BitNet-like architectures as smooth quantization parameters approach zero. We establish that empirical measures of latent weights converge weakly to solutions of…

The general model of coagulation is considered. For basic classes of unbounded coagulation kernels the central limit theorem (CLT) is obtained for the fluctuations around the dynamic law of large numbers (LLN). A rather precise rate of…

Motivated by the increasing use of kernel-based metrics for high-dimensional and large-scale data, we study the asymptotic behavior of kernel two-sample tests when the dimension and sample sizes both diverge to infinity. We focus on the…

We provide non-asymptotic convergence rates of the Polyak-Ruppert averaged stochastic gradient descent (SGD) to a normal random vector for a class of twice-differentiable test functions. A crucial intermediate step is proving a…