照我做，而非照我说：将语言 grounding 于机器人可供性
摘要
大语言模型能够编码关于世界的丰富语义知识。此类知识对于旨在执行用自然语言表述的高级、时序扩展指令的机器人而言可能极为有用。然而，语言模型的一个显著弱点是缺乏真实世界经验，这使其难以在特定具身中用于决策。例如，要求语言模型描述如何清理洒出的液体可能产生合理的叙述，但可能不适用于需要在特定环境中执行该任务的特定智能体（如机器人）。我们提出通过预训练技能提供真实世界 grounding，用于约束模型提出既可行又符合语境的自然语言动作。机器人可充当语言模型的“手与眼”，而语言模型提供关于任务的高级语义知识。我们展示了如何将低级技能与大语言模型结合，使语言模型提供关于执行复杂且时序扩展指令流程的高级知识，而这些技能相关的价值函数提供将该知识连接到特定物理环境所需的 grounding。我们在多个真实世界机器人任务上评估了我们的方法，展示了真实世界 grounding 的必要性，并且该方法能够在移动 manipulator 上完成长时序、抽象的自然语言指令。项目网站和视频可在 https://say-can.github.io/ 找到。
引用
@article{arxiv.2204.01691,
title = {Do As I Can, Not As I Say: Grounding Language in Robotic Affordances},
author = {Michael Ahn and Anthony Brohan and Noah Brown and Yevgen Chebotar and Omar Cortes and Byron David and Chelsea Finn and Chuyuan Fu and Keerthana Gopalakrishnan and Karol Hausman and Alex Herzog and Daniel Ho and Jasmine Hsu and Julian Ibarz and Brian Ichter and Alex Irpan and Eric Jang and Rosario Jauregui Ruano and Kyle Jeffrey and Sally Jesmonth and Nikhil J Joshi and Ryan Julian and Dmitry Kalashnikov and Yuheng Kuang and Kuang-Huei Lee and Sergey Levine and Yao Lu and Linda Luu and Carolina Parada and Peter Pastor and Jornell Quiambao and Kanishka Rao and Jarek Rettinghouse and Diego Reyes and Pierre Sermanet and Nicolas Sievers and Clayton Tan and Alexander Toshev and Vincent Vanhoucke and Fei Xia and Ted Xiao and Peng Xu and Sichun Xu and Mengyuan Yan and Andy Zeng},
journal= {arXiv preprint arXiv:2204.01691},
year = {2022}
}
备注
See website at https://say-can.github.io/ V1. Initial Upload. V2. Added PaLM results. Added study about new capabilities (drawer manipulation, chain of thought prompting, multilingual instructions). Added an ablation study of language model size. Added an open-source version of \algname on a simulated tabletop environment. Improved readability