Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Abstract
We show that passing input points through a simple Fourier feature mapping enables a multilayer perceptron (MLP) to learn high-frequency functions in low-dimensional problem domains. These results shed light on recent advances in computer vision and graphics that achieve state-of-the-art results by using MLPs to represent complex 3D objects and scenes. Using tools from the neural tangent kernel (NTK) literature, we show that a standard MLP fails to learn high frequencies both in theory and in practice. To overcome this spectral bias, we use a Fourier feature mapping to transform the effective NTK into a stationary kernel with a tunable bandwidth. We suggest an approach for selecting problem-specific Fourier features that greatly improves the performance of MLPs for low-dimensional regression tasks relevant to the computer vision and graphics communities.
Cite
@article{arxiv.2006.10739,
title = {Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains},
author = {Matthew Tancik and Pratul P. Srinivasan and Ben Mildenhall and Sara Fridovich-Keil and Nithin Raghavan and Utkarsh Singhal and Ravi Ramamoorthi and Jonathan T. Barron and Ren Ng},
journal= {arXiv preprint arXiv:2006.10739},
year = {2020}
}
Comments
Project page: https://people.eecs.berkeley.edu/~bmild/fourfeat/