Les Houches Lectures on Deep Learning at Large & Infinite Width
Abstract
These lectures, presented at the 2022 Les Houches Summer School on Statistical Physics and Machine Learning, focus on the infinite-width limit and large-width regime of deep neural networks. Topics covered include various statistical and dynamical properties of these networks. In particular, the lecturers discuss properties of random deep neural networks; connections between trained deep neural networks, linear models, kernels, and Gaussian processes that arise in the infinite-width limit; and perturbative and non-perturbative treatments of large but finite-width networks, at initialization and after training.
Keywords
Cite
@article{arxiv.2309.01592,
title = {Les Houches Lectures on Deep Learning at Large & Infinite Width},
author = {Yasaman Bahri and Boris Hanin and Antonin Brossollet and Vittorio Erba and Christian Keup and Rosalba Pacelli and James B. Simon},
journal= {arXiv preprint arXiv:2309.01592},
year = {2024}
}
Comments
These are notes from lectures delivered by Yasaman Bahri and Boris Hanin at the 2022 Les Houches Summer School on Statistics Physics and Machine Learning and a first version of them were transcribed by Antonin Brossollet, Vittorio Erba, Christian Keup, Rosalba Pacelli, James B. Simon