Structures of Neural Network Effective Theories
High Energy Physics - Theory
2023-05-12 v1 Disordered Systems and Neural Networks
Machine Learning
High Energy Physics - Phenomenology
Machine Learning
Abstract
We develop a diagrammatic approach to effective field theories (EFTs) corresponding to deep neural networks at initialization, which dramatically simplifies computations of finite-width corrections to neuron statistics. The structures of EFT calculations make it transparent that a single condition governs criticality of all connected correlators of neuron preactivations. Understanding of such EFTs may facilitate progress in both deep learning and field theory simulations.
Cite
@article{arxiv.2305.02334,
title = {Structures of Neural Network Effective Theories},
author = {Ian Banta and Tianji Cai and Nathaniel Craig and Zhengkang Zhang},
journal= {arXiv preprint arXiv:2305.02334},
year = {2023}
}
Comments
7+13 pages, 5 figures