Related papers: A Dynamical Central Limit Theorem for Shallow Neur…
A longstanding goal in deep learning research has been to precisely characterize training and generalization. However, the often complex loss landscapes of neural networks have made a theory of learning dynamics elusive. In this work, we…
We consider optimizing two-layer neural networks in the mean-field regime where the learning dynamics of network weights can be approximated by the evolution in the space of probability measures over the weight parameters associated with…
The aim of this paper is to study the asymptotic expansion in total variation in the Central Limit Theorem when the law of the basic random variable is locally lower-bounded by the Lebesgue measure (or equivalently, has an absolutely…
Fr\'echet means of samples from a probability measure $\mu$ on any smoothly stratified metric space M with curvature bounded above are shown to satisfy a central limit theorem (CLT). The methods and results proceed by introducing and…
In this article, we try to give an answer to the simple question: ``\textit{What is the critical growth rate of the dimension $p$ as a function of the sample size $n$ for which the Central Limit Theorem holds uniformly over the collection…
A central limit theorem is established for a sum of random variables belonging to a sequence of random fields. The fields are assumed to have zero mean conditional on the past history and to satisfy certain conditional $\alpha$-mixing…
We study the gradient-based training of large-depth residual networks (ResNets) from standard random initializations. We show that infinite-depth ResNets behave as if they were infinitely wide, regardless of their actual width. More…
Existing analyses of neural network training often operate under the unrealistic assumption of an extremely small learning rate. This lies in stark contrast to practical wisdom and empirical studies, such as the work of J. Cohen et al.…
Finding parameters in a deep neural network (NN) that fit training data is a nonconvex optimization problem, but a basic first-order optimization method (gradient descent) finds a global optimizer with perfect fit (zero-loss) in many…
The dynamics of gradient-based training in neural networks often exhibit nontrivial structures; hence, understanding them remains a central challenge in theoretical machine learning. In particular, a concept of feature unlearning, in which…
The key to generalization is controlling the complexity of the network. However, there is no obvious control of complexity -- such as an explicit regularization term -- in the training of deep networks for classification. We will show that…
We prove the central limit theorem (CLT) for a sequence of independent zero-mean random variables $\xi_j$, perturbed by predictable multiplicative factors $\lambda_j$ with values in intervals $[\underline\lambda_j,\overline\lambda_j]$. It…
We establish central and non-central limit theorems for sequences of functionals of the Gaussian output of an infinitely-wide random neural network on the d-dimensional sphere . We show that the asymptotic behaviour of these functionals as…
There has been remarkable progress over the past decade in establishing finite-sample, non-asymptotic bounds on recovering unknown system parameters from observed system behavior. Surprisingly, however, we show that the current…
Neural networks are widely used to approximate unknown functions in control. A common neural network architecture uses a single hidden layer (i.e. a shallow network), in which the input parameters are fixed in advance and only the output…
We investigate the probability density of rescaled sums of iterates of deterministic dynamical systems, a problem relevant for many complex physical systems consisting of dependent random variables. A Central Limit Theorem (CLT) is only…
Random spatial networks-that is, graphs whose connectivity is governed by geometric proximity-have emerged as fundamental models for systems constrained by an underlying spatial structure. A prototypical example is the random geometric…
Confidence intervals based on the central limit theorem (CLT) are a cornerstone of classical statistics. Despite being only asymptotically valid, they are ubiquitous because they permit statistical inference under weak assumptions and can…
This paper proposes an asymptotic theory for online inference of the stochastic gradient descent (SGD) iterates with dropout regularization in linear regression. Specifically, we establish the geometric-moment contraction (GMC) for constant…
A surprising phenomenon in the training of neural networks is the ability of gradient descent to find global minimizers of the training loss despite its non-convexity. Following earlier works, we investigate this behavior for wide shallow…