Grounding Open-Domain Instructions to Automate Web Support Tasks
Abstract
Grounding natural language instructions on the web to perform previously unseen tasks enables accessibility and automation. We introduce a task and dataset to train AI agents from open-domain, step-by-step instructions originally written for people. We build RUSS (Rapid Universal Support Service) to tackle this problem. RUSS consists of two models: First, a BERT-LSTM with pointers parses instructions to ThingTalk, a domain-specific language we design for grounding natural language on the web. Then, a grounding model retrieves the unique IDs of any webpage elements requested in ThingTalk. RUSS may interact with the user through a dialogue (e.g. ask for an address) or execute a web operation (e.g. click a button) inside the web runtime. To augment training, we synthesize natural language instructions mapped to ThingTalk. Our dataset consists of 80 different customer service problems from help websites, with a total of 741 step-by-step instructions and their corresponding actions. RUSS achieves 76.7% end-to-end accuracy predicting agent actions from single instructions. It outperforms state-of-the-art models that directly map instructions to actions without ThingTalk. Our user study shows that RUSS is preferred by actual users over web navigation.
Keywords
Cite
@article{arxiv.2103.16057,
title = {Grounding Open-Domain Instructions to Automate Web Support Tasks},
author = {Nancy Xu and Sam Masling and Michael Du and Giovanni Campagna and Larry Heck and James Landay and Monica S Lam},
journal= {arXiv preprint arXiv:2103.16057},
year = {2021}
}
Comments
To be published in NAACL 2021