相关论文: Mean Field Analysis of Neural Networks: A Central …

We analyze multi-layer neural networks in the asymptotic regime of simultaneously (A) large network sizes and (B) large numbers of stochastic gradient descent training iterations. We rigorously establish the limiting behavior of the…

Recent theoretical works have characterized the dynamics of wide shallow neural networks trained via gradient descent in an asymptotic mean-field limit when the width tends towards infinity. At initialization, the random sampling of the…

We consider learning two layer neural networks using stochastic gradient descent. The mean-field description of this learning dynamics approximates the evolution of the network weights by an evolution in the space of probability…

We consider a system of $N$ neurons, each spiking randomly with rate depending on its membrane potential. When a neuron spikes, its potential is reset to $0$ and all other neurons receive an additional amount $h/N$ of potential, where $ h >…

A central limit theorem is established for a sum of random variables belonging to a sequence of random fields. The fields are assumed to have zero mean conditional on the past history and to satisfy certain conditional $\alpha$-mixing…

We prove a Quantitative Functional Central Limit Theorem for one-hidden-layer neural networks with generic activation function. The rates of convergence that we establish depend heavily on the smoothness of the activation function, and they…

We consider optimizing two-layer neural networks in the mean-field regime where the learning dynamics of network weights can be approximated by the evolution in the space of probability measures over the weight parameters associated with…

Our main results are quantitative bounds in the multivariate normal approximation of centred subgraph counts in random graphs generated by a general graphon and independent vertex labels. We are interested in these statistics because they…

The convergence of stochastic interacting particle systems in the mean-field limit to solutions of conservative stochastic partial differential equations is established, with optimal rate of convergence. As a second main result, a…

This work develops a mean-field analysis for the asymptotic behavior of deep BitNet-like architectures as smooth quantization parameters approach zero. We establish that empirical measures of latent weights converge weakly to solutions of…

We study the asymptotic shape of the trajectory of the stochastic gradient descent algorithm applied to a convex objective function. Under mild regularity assumptions, we prove a functional central limit theorem for the properly rescaled…

In this work, we obtain the central limit theorem for fluctuations of Young diagrams around their limit shape in the bulk of the "spectrum" of partitions of a large integer n (under the Plancherel measure). More specifically, we show that,…

Machine learning, and in particular neural network models, have revolutionized fields such as image, text, and speech recognition. Today, many important real-world applications in these areas are driven by neural networks. There are also…

In the mean field regime, neural networks are appropriately scaled so that as the width tends to infinity, the learning dynamics tends to a nonlinear and nontrivial dynamical limit, known as the mean field limit. This lends a way to study…

We study the mean-field limit of a generic class of dynamic co-evolving latent space networks motivated by the social and opinion dynamics literature. Such models include $n$ agents, whose opinions are given by latent stochastic processes,…

Several differential equation models have been proposed to explain the formation of patterns characteristic of the grid cell network. Understanding the robustness of these patterns with respect to noise is one of the key open questions in…

We study large but finite neural networks that, in the thermodynamic limit, admit an exact low-dimensional mean-field description. We assume that the governing mean-field equations describing macroscopic quantities such as the mean firing…

The stunning empirical successes of neural networks currently lack rigorous theoretical explanation. What form would such an explanation take, in the face of existing complexity-theoretic lower bounds? A first step might be to show that…

The mean field (MF) theory of multilayer neural networks centers around a particular infinite-width scaling, where the learning dynamics is closely tracked by the MF limit. A random fluctuation around this infinite-width limit is expected…

Nowadays, neural networks are widely used in many applications as artificial intelligence models for learning tasks. Since typically neural networks process a very large amount of data, it is convenient to formulate them within the…