Related papers: A Dynamical Central Limit Theorem for Shallow Neur…

Multi-layer feedforward networks have been used to approximate a wide range of nonlinear functions. An important and fundamental problem is to understand the learnability of a network model through its statistical risk, or the expected…

Cross-domain crowd counting (CDCC) is a hot topic due to its importance in public safety. The purpose of CDCC is to alleviate the domain shift between the source and target domain. Recently, typical methods attempt to extract…

Recent theoretical results show that gradient descent on deep neural networks under exponential loss functions locally maximizes classification margin, which is equivalent to minimizing the norm of the weight matrices under margin…

Establishing central limit theorems (CLTs) for ergodic averages of Markov chains is a fundamental problem in probability and its applications. Since the seminal work~\cite{MR834478}, a vast literature has emerged on the sufficient…

Modern machine learning models are typically trained via multi-pass stochastic gradient descent (SGD) with small batch sizes, and understanding their dynamics in high dimensions is of great interest. However, an analytical framework for…

We develop a new toolbox for the analysis of the global behavior of stochastic discrete particle systems. We introduce and study the notion of the Schur generating function of a random discrete configuration. Our main result provides a…

We investigate the generalization and optimization properties of shallow neural-network classifiers trained by gradient descent in the interpolating regime. Specifically, in a realizable scenario where model weights can achieve arbitrarily…

In this paper we propose a new approach to the central limit theorem (CLT), based on functions of bounded F\'echet variation for the continuously differentiable linear statistics of random matrix ensembles which relies on: a weaker form of…

We establish central limit theorems for the Sample Average Approximation (SAA) method in discrete-time, finite-horizon stochastic optimal control. Our analysis is based on an abstract limit theorem for stochastic backward recursions, which…

Dynamical systems theory has recently been applied in optimization to prove that gradient descent algorithms bypass so-called strict saddle points of the loss function. However, in many modern machine learning applications, the required…

We establish a central limit theorem (CLT) for families of products of $\epsilon$-independent random variables. We utilize graphon limits to encode the evolution of independence and characterize the limiting distribution. Our framework…

We prove linear convergence of gradient descent to a global optimum for the training of deep residual networks with constant layer width and smooth activation function. We show that if the trained weights, as a function of the layer index,…

For a L\'evy basis $L$ on $\mathbb{R}^d$ and a suitable kernel function $f:\mathbb{R}^d \to \mathbb{R}$, consider the continuous spatial moving average field $X=(X_t)_{t\in \mathbb{R}^d}$ defined by $X_t = \int_{\mathbb{R}^d} f(t-s) \,…

In this paper we consider the asymptotic distributions of functionals of the sample covariance matrix and the sample mean vector obtained under the assumption that the matrix of observations has a matrix-variate location mixture of normal…

We establish a central limit theorem for the fluctuations of the linear statistics in the $\beta$-ensemble of dimension $N$ at a temperature proportional to $N$ and with confining smooth potential. In this regime, the particles do not…

We consider deterministic random walks on the real line driven by irrational rotations, or equivalently, skew product extensions of a rotation by $\alpha$ where the skewing cocycle is a piecewise constant mean zero function with a jump by…

Understanding the asymptotic behavior of gradient-descent training of deep neural networks is essential for revealing inductive biases and improving network performance. We derive the infinite-time training limit of a mathematically…

Numerous researches have proved that deep neural networks (DNNs) can fit everything in the end even given data with noisy labels, and result in poor generalization performance. However, recent studies suggest that DNNs tend to gradually…

Deep neural networks (DNNs) have significantly advanced machine learning, with model depth playing a central role in their successes. The dynamical system modeling approach has recently emerged as a powerful framework, offering new…

There has been growing interest in generalization performance of large multilayer neural networks that can be trained to achieve zero training error, while generalizing well on test data. This regime is known as 'second descent' and it…