Conditionally Combining Robot Skills using Large Language Models
Abstract
This paper combines two contributions. First, we introduce an extension of the Meta-World benchmark, which we call "Language-World," which allows a large language model to operate in a simulated robotic environment using semi-structured natural language queries and scripted skills described using natural language. By using the same set of tasks as Meta-World, Language-World results can be easily compared to Meta-World results, allowing for a point of comparison between recent methods using Large Language Models (LLMs) and those using Deep Reinforcement Learning. Second, we introduce a method we call Plan Conditioned Behavioral Cloning (PCBC), that allows finetuning the behavior of high-level plans using end-to-end demonstrations. Using Language-World, we show that PCBC is able to achieve strong performance in a variety of few-shot regimes, often achieving task generalization with as little as a single demonstration. We have made Language-World available as open-source software at https://github.com/krzentner/language-world/.
Cite
@article{arxiv.2310.17019,
title = {Conditionally Combining Robot Skills using Large Language Models},
author = {K. R. Zentner and Ryan Julian and Brian Ichter and Gaurav S. Sukhatme},
journal= {arXiv preprint arXiv:2310.17019},
year = {2023}
}