Representation formulas and pointwise properties for Barron functions
Abstract
We study the natural function space for infinitely wide two-layer neural networks with ReLU activation (Barron space) and establish different representation formulae. In two cases, we describe the space explicitly up to isomorphism. Using a convenient representation, we study the pointwise properties of two-layer networks and show that functions whose singular set is fractal or curved (for example distance functions from smooth submanifolds) cannot be represented by infinitely wide two-layer networks with finite path-norm. We use this structure theorem to show that the only -diffeomorphisms which Barron space are affine. Furthermore, we show that every Barron function can be decomposed as the sum of a bounded and a positively one-homogeneous function and that there exist Barron functions which decay rapidly at infinity and are globally Lebesgue-integrable. This result suggests that two-layer neural networks may be able to approximate a greater variety of functions than commonly believed.
Keywords
Cite
@article{arxiv.2006.05982,
title = {Representation formulas and pointwise properties for Barron functions},
author = {Weinan E and Stephan Wojtowytsch},
journal= {arXiv preprint arXiv:2006.05982},
year = {2021}
}