Related papers: A Dynamical Central Limit Theorem for Shallow Neur…
In this paper, we study second order fluctuations for the size of the range of a critical branching random walk (BRW) in $\mathbb Z^d$. We consider the BRW with geometric offspring indexed by the Kesten tree, and show that the size of its…
We obtain large deviations estimates for both sequential and random compositions of intermittent maps. We also address the question of whether or not centering is necessary for the quenched central limit theorems (CLT) obtained by Nicol,…
The \textit{Central Limit Theorem (CLT)} is at the heart of a great deal of applied problem-solving in statistics and data science, but the theorem is silent on an important implementation issue: \textit{how much data do you need for the…
We study the relationship between chaotic behavior and the Central Limit Theorem (CLT) in the Kuramoto model. We calculate sums of angles at equidistant times along deterministic trajectories of single oscillators and we show that, when…
Deep neural networks achieve stellar generalisation even when they have enough parameters to easily fit all their training data. We study this phenomenon by analysing the dynamics and the performance of over-parameterised two-layer neural…
Continual learning (CL) presents a fundamental challenge in training neural networks on sequential tasks without experiencing catastrophic forgetting. Traditionally, the dominant approach in CL has been gradient-based optimization, where…
In this paper we exhibit new classes of smooth systems which satisfy the Central Limit Theorem (CLT) and have (at least) one of the following properties: (1) zero entropy; (2) weak but not strong mixing; (3) (polynomially) mixing but not…
In this work, a novel and model-based artificial neural network (ANN) training method is developed supported by optimal control theory. The method augments training labels in order to robustly guarantee training loss convergence and improve…
Learning a task induces connectivity changes in neural circuits, thereby changing their dynamics. To elucidate task related neural dynamics we study trained Recurrent Neural Networks. We develop a Mean Field Theory for Reservoir Computing…
We analyze in a closed form the learning dynamics of stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides…
This dissertation studies a fundamental open challenge in deep learning theory: why do deep networks generalize well even while being overparameterized, unregularized and fitting the training data to zero error? In the first part of the…
Vanishing (and exploding) gradients effect is a common problem for recurrent neural networks with nonlinear activation functions which use backpropagation method for calculation of derivatives. Deep feedforward neural networks with many…
In a vast area of probabilistic limit theorems for dynamical systems with chaotic behaviors always only functional form (exponential, power, etc) of the asymptotic laws and of convergence rates were studied. However, for basically all…
The objective of this study is to investigate the limiting behavior of a subgraph counting process. The subgraph counting process we consider counts the number of subgraphs having a specific shape that exist outside an expanding ball as the…
We consider Betti numbers of the excursion of a smooth Euclidean Gaussian field restricted to a rectangular window, in the asymptotics where the window grows to R^d . With motivations coming from Topological Data Analysis, we derive a…
In previous papers, we studied the asymptotic behaviour of $S_N(A,X)=(2N+1)^{-d/2}\sum_{n \in A_N} X_n,$ where $X$ is a centered, stationary and weakly dependent random field, and $A_N=A \cap [-N,N]^d$, $A \subset \mathbb{Z}^d$. This leads…
The paper concerns the $d$-dimensional stochastic approximation recursion, $$ \theta_{n+1}= \theta_n + \alpha_{n + 1} f(\theta_n, \Phi_{n+1}) $$ where $ \{ \Phi_n \}$ is a stochastic process on a general state space, satisfying a…
Neural networks have been shown to perform incredibly well in classification tasks over structured high-dimensional datasets. However, the learning dynamics of such networks is still poorly understood. In this paper we study in detail the…
This article studies the dynamics of the mean-field approximation of continuous random networks. These networks are stochastic integrodifferential equations driven by Gaussian noise. The kernels in the integral operators are realizations of…
Central limit theorems (CLTs) have a long history in probability and statistics. They play a fundamental role in constructing valid statistical inference procedures. Over the last century, various techniques have been developed in…