Tuning Universality in Deep Neural Networks
Disordered Systems and Neural Networks
2025-12-02 v1 Statistical Mechanics
Artificial Intelligence
Adaptation and Self-Organizing Systems
Biological Physics
Abstract
Deep neural networks (DNNs) exhibit crackling-like avalanches whose origin lacks a mechanistic explanation. Here, I derive a stochastic theory of deep information propagation (DIP) by incorporating Central Limit Theorem (CLT)-level fluctuations. Four effective couplings characterize the dynamics, yielding a Landau description of the static exponents and a Directed Percolation (DP) structure of activity cascades. Tuning the couplings selects between avalanche dynamics generated by a Brownian Motion (BM) in a logarithmic trap and an absorbed free BM, each corresponding to a distinct universality classes. Numerical simulations confirm the theory and demonstrate that activation function design controls the collective dynamics in random DNNs.
Cite
@article{arxiv.2512.00168,
title = {Tuning Universality in Deep Neural Networks},
author = {Arsham Ghavasieh},
journal= {arXiv preprint arXiv:2512.00168},
year = {2025}
}