On the Approximation Power of Two-Layer Networks of Random ReLUs
Machine Learning
2021-09-09 v2 Neural and Evolutionary Computing
Machine Learning
Abstract
This paper considers the following question: how well can depth-two ReLU networks with randomly initialized bottom-level weights represent smooth functions? We give near-matching upper- and lower-bounds for -approximation in terms of the Lipschitz constant, the desired accuracy, and the dimension of the problem, as well as similar results in terms of Sobolev norms. Our positive results employ tools from harmonic analysis and ridgelet representation theory, while our lower-bounds are based on (robust versions of) dimensionality arguments.
Keywords
Cite
@article{arxiv.2102.02336,
title = {On the Approximation Power of Two-Layer Networks of Random ReLUs},
author = {Daniel Hsu and Clayton Sanford and Rocco A. Servedio and Emmanouil-Vasileios Vlatakis-Gkaragkounis},
journal= {arXiv preprint arXiv:2102.02336},
year = {2021}
}
Comments
39 pages, COLT version