This paper describes a strategy for implementing a robotic system capable of performing General Purpose Service Robot (GPSR) tasks in robocup@home. The GPSR task is that a real robot hears a variety of commands in spoken language and executes a task in a daily life environment. To achieve the task, we integrate foundation models based inference system and a state machine task executable. The foundation models plan the task and detect objects with open vocabulary, and a state machine task executable manages each robot's actions. This system works stable, and we took first place in the RoboCup@home Japan Open 2022's GPSR with 130 points, more than 85 points ahead of the other teams.
Cite
@article{arxiv.2308.03357,
title = {Foundation Model based Open Vocabulary Task Planning and Executive System for General Purpose Service Robots},
author = {Yoshiki Obinata and Naoaki Kanazawa and Kento Kawaharazuka and Iori Yanokura and Soonhyo Kim and Kei Okada and Masayuki Inaba},
journal= {arXiv preprint arXiv:2308.03357},
year = {2023}
}