We present a program synthesis-oriented dataset consisting of human written problem statements and solutions for these problems. The problem statements were collected via crowdsourcing and the program solutions were extracted from human-written solutions in programming competitions, accompanied by input/output examples. We propose using this dataset for the program synthesis tasks aimed for working with real user-generated data. As a baseline we present few models, with the best model achieving 8.8% accuracy, showcasing both the complexity of the dataset and large room for future research.
@article{arxiv.1807.03168,
title = {NAPS: Natural Program Synthesis Dataset},
author = {Maksym Zavershynskyi and Alex Skidanov and Illia Polosukhin},
journal= {arXiv preprint arXiv:1807.03168},
year = {2018}
}
Comments
4 pages, 5 tables in 2nd Workshop on Neural Abstract Machines & Program Induction (NAMPI), @ICML 2018