Why and When Deep is Better than Shallow: Implementation-Agnostic State-Transition Model of Deep Learning
Abstract
Why and when does depth improve generalization? We study this question in an implementation-agnostic state-transition model, where a depth- predictor is a readout class composed with the word ball generated by hidden state transitions. Generalization bounds separate implementation error, approximation error, and statistical complexity, and upper bound the depth-dependent variance term by a Dudley entropy integral over , with a conditional lower-bound diagnostic under readout separation. We identify geometric and semigroup mechanisms that keep this entropy contribution saturated or polynomial, and contrast them with separation mechanisms that recover the classical exponential-growth obstruction. Coupling these variance upper bounds with approximation rates gives typical depth trade-off patterns, clarifying that depth is statistically favorable when approximation improves rapidly while the transition semigroup remains geometrically tame.
Cite
@article{arxiv.2505.15064,
title = {Why and When Deep is Better than Shallow: Implementation-Agnostic State-Transition Model of Deep Learning},
author = {Sho Sonoda and Yuka Hashimoto and Isao Ishikawa and Masahiro Ikeda},
journal= {arXiv preprint arXiv:2505.15064},
year = {2026}
}