Neural networks: deep, shallow, or in between?
Machine Learning
2023-10-12 v1 Machine Learning
Numerical Analysis
Numerical Analysis
Abstract
We give estimates from below for the error of approximation of a compact subset from a Banach space by the outputs of feed-forward neural networks with width W, depth l and Lipschitz activation functions. We show that, modulo logarithmic factors, rates better that entropy numbers' rates are possibly attainable only for neural networks for which the depth l goes to infinity, and that there is no gain if we fix the depth and let the width W go to infinity.
Keywords
Cite
@article{arxiv.2310.07190,
title = {Neural networks: deep, shallow, or in between?},
author = {Guergana Petrova and Przemyslaw Wojtaszczyk},
journal= {arXiv preprint arXiv:2310.07190},
year = {2023}
}