Functional Large Deviations for Wide Deep Neural Networks with Gaussian Initialization and Lipschitz Activations
Probability
2026-01-27 v1 Statistics Theory
Statistics Theory
Abstract
We establish a functional large deviation principle for fully connected multi-layer perceptrons with i.i.d. Gaussian weights (LeCun initialization) and general Lipschitz activation functions, including therefore the popular case of ReLU. The large deviation principle holds for the entire network output process on any compact input set. The proof combines exponential tightness for recursively defined processes, finite-dimensional large deviations, and the Dawson-G\"artner theorem, extending existing results beyond finite input sets and less general activations.
Keywords
Cite
@article{arxiv.2601.18276,
title = {Functional Large Deviations for Wide Deep Neural Networks with Gaussian Initialization and Lipschitz Activations},
author = {Claudio Macci and Barbara Pacchiarotti and Katerina Papagiannouli and Giovanni Luca Torrisi and Dario Trevisan},
journal= {arXiv preprint arXiv:2601.18276},
year = {2026}
}