Symbolic equations are at the core of scientific discovery. The task of discovering the underlying equation from a set of input-output pairs is called symbolic regression. Traditionally, symbolic regression methods use hand-designed strategies that do not improve with experience. In this paper, we introduce the first symbolic regression method that leverages large scale pre-training. We procedurally generate an unbounded set of equations, and simultaneously pre-train a Transformer to predict the symbolic equation from a corresponding set of input-output-pairs. At test time, we query the model on a new set of points and use its output to guide the search for the equation. We show empirically that this approach can re-discover a set of well-known physical equations, and that it improves over time with more data and compute.
@article{arxiv.2106.06427,
title = {Neural Symbolic Regression that Scales},
author = {Luca Biggio and Tommaso Bendinelli and Alexander Neitz and Aurelien Lucchi and Giambattista Parascandolo},
journal= {arXiv preprint arXiv:2106.06427},
year = {2021}
}
Comments
Accepted at the 38th International Conference on Machine Learning (ICML) 2021