Related papers: A Dynamical Central Limit Theorem for Shallow Neur…
We undertake a precise study of the asymptotic and non-asymptotic properties of stochastic approximation procedures with Polyak-Ruppert averaging for solving a linear system $\bar{A} \theta = \bar{b}$. When the matrix $\bar{A}$ is Hurwitz,…
We develop a mean-field theory of dropout as a perturbation of critical signal propagation at the edge of chaos. Dropout shifts the perfect-alignment fixed point, making the depth scale for information propagation finite even at critical…
We develop a mean field theory for batch normalization in fully-connected feedforward neural networks. In so doing, we provide a precise characterization of signal propagation and gradient backpropagation in wide batch-normalized networks…
While significant theoretical progress has been achieved, unveiling the generalization mystery of overparameterized neural networks still remains largely elusive. In this paper, we study the generalization behavior of shallow neural…
The batch means estimator of the MCMC variance is a simple and effective measure of accuracy for MCMC based ergodic averages. Under various regularity conditions, the estimator has been shown to be consistent for the true variance. However,…
We study random dynamical systems composed of LSV maps with varying parameters, without any mixing assumptions on the base space of random dynamics. We establish a quenched central limit theorem and identify conditions under which the…
We study the training dynamics of shallow neural networks, in a two-timescale regime in which the stepsizes for the inner layer are much smaller than those for the outer layer. In this regime, we prove convergence of the gradient flow to a…
Neural Collapse is a phenomenon where the last-layer representations of a well-trained neural network converge to a highly structured geometry. In this paper, we focus on its first (and most basic) property, known as NC1: the within-class…
Real-world networks tend to be scale free, having heavy-tailed degree distributions with more hubs than predicted by classical random graph generation methods. Preferential attachment and growth are the most commonly accepted mechanisms…
Training large neural networks exposes neural scaling laws for the generalization error, which points to a universal behavior across network architectures of learning in high dimensions. It was also shown that this effect persists in the…
This paper investigates the asymptotic behavior of the deterministic and stochastic Cram\'er-Rao Bounds (CRB) for semi-blind channel estimation in massive multiple-input multiple-output (MIMO) systems. We derive and analyze mathematically…
We prove quenched versions of (i) a large deviations principle (LDP), (ii) a central limit theorem (CLT), and (iii) a local central limit theorem (LCLT) for non-autonomous dynamical systems. A key advance is the extension of the spectral…
We prove, for any $\beta >0$, a central limit theorem for the fluctuations of linear statistics in the Sine-$\beta$ process, which is the infinite volume limit of the random microscopic behavior in the bulk of one-dimensional log-gases at…
In this paper we establish a general dynamical Central Limit Theorem (CLT) for group actions which are exponentially mixing of all orders. In particular, the main result applies to Cartan flows on finite-volume quotients of simple Lie…
We investigate the asymptotic risk of a general class of overparameterized likelihood models, including deep models. The recent empirical success of large-scale models has motivated several theoretical studies to investigate a scenario…
The classical statistical learning theory implies that fitting too many parameters leads to overfitting and poor performance. That modern deep neural networks generalize well despite a large number of parameters contradicts this finding and…
`Double descent' delineates the generalization behaviour of models depending on the regime they belong to: under- or over-parameterized. The current theoretical understanding behind the occurrence of this phenomenon is primarily based on…
The Central Limit Theorem (CLT) for additive functionals of Markov chains is a well known result with a long history. In this paper we present applications to two finite-memory versions of the Elephant Random Walk, solving a problem from…
The law of large numbers (LLN) and central limit theorem (CLT) are long and widely been known as two fundamental results in probability theory. Recently problems of model uncertainties in statistics, measures of risk and superhedging in…
We consider stochastic approximations of sampling algorithms, such as Stochastic Gradient Langevin Dynamics (SGLD) and the Random Batch Method (RBM) for Interacting Particle Dynamcs (IPD). We observe that the noise introduced by the…