Gemini Robotics: 将 AI 引入物理世界
机器人学
2025-03-27 v1
摘要
最近的大多模态模型的进展导致在数字领域出现了显著的通用能力，但将其转化为物理智能体（如机器人）仍是一个重大挑战。本报告介绍了一个新的 AI 模型家族，专为机器人设计并基于 Gemini 2.0 构建。我们 presented Gemini Robotics，一种先进的视觉-语言-动作（VLA）通用模型，能够直接控制机器人。Gemini Robotics 在处理各种复杂操作任务方面能够执行平滑且具有反应性的动作，同时对对象类型和位置的变化具有鲁棒性，能够处理未遇到的环境并遵循多样化的开放词汇指令。我们展示，经过额外微调后，Gemini Robotics 可以专用于新能力，包括解决长期、高度灵巧的任务、仅需 100 次演示即可学习新任务以及适应完全新的机器人本体。这是可能的原因是 Gemini Robotics 建立在 Gemini Robotics-ER 模型之上，该模型是本 work 介绍的第二个模型。Gemini Robotics-ER（具身推理）将 Gemini 的多模态推理能力扩展到物理世界，具有增强的空间和时间理解。这使得与机器人相关的能力得以实现，包括目标检测、指向、轨迹和抓取预测，以及多视角对应和 3D 边界框预测。我们展示了这种新颖组合如何支持 various 机器人应用。我们也讨论并解决了与 this new class robotics foundation models 相关的重要安全考虑。Gemini Robotics 家族标志着向开发实现 AI 在物理世界潜力的通用机器人迈出了重大步伐。
引用
@article{arxiv.2503.20020,
title = {Gemini Robotics: Bringing AI into the Physical World},
author = {Gemini Robotics Team and Saminda Abeyruwan and Joshua Ainslie and Jean-Baptiste Alayrac and Montserrat Gonzalez Arenas and Travis Armstrong and Ashwin Balakrishna and Robert Baruch and Maria Bauza and Michiel Blokzijl and Steven Bohez and Konstantinos Bousmalis and Anthony Brohan and Thomas Buschmann and Arunkumar Byravan and Serkan Cabi and Ken Caluwaerts and Federico Casarini and Oscar Chang and Jose Enrique Chen and Xi Chen and Hao-Tien Lewis Chiang and Krzysztof Choromanski and David D'Ambrosio and Sudeep Dasari and Todor Davchev and Coline Devin and Norman Di Palo and Tianli Ding and Adil Dostmohamed and Danny Driess and Yilun Du and Debidatta Dwibedi and Michael Elabd and Claudio Fantacci and Cody Fong and Erik Frey and Chuyuan Fu and Marissa Giustina and Keerthana Gopalakrishnan and Laura Graesser and Leonard Hasenclever and Nicolas Heess and Brandon Hernaez and Alexander Herzog and R. Alex Hofer and Jan Humplik and Atil Iscen and Mithun George Jacob and Deepali Jain and Ryan Julian and Dmitry Kalashnikov and M. Emre Karagozler and Stefani Karp and Chase Kew and Jerad Kirkland and Sean Kirmani and Yuheng Kuang and Thomas Lampe and Antoine Laurens and Isabel Leal and Alex X. Lee and Tsang-Wei Edward Lee and Jacky Liang and Yixin Lin and Sharath Maddineni and Anirudha Majumdar and Assaf Hurwitz Michaely and Robert Moreno and Michael Neunert and Francesco Nori and Carolina Parada and Emilio Parisotto and Peter Pastor and Acorn Pooley and Kanishka Rao and Krista Reymann and Dorsa Sadigh and Stefano Saliceti and Pannag Sanketi and Pierre Sermanet and Dhruv Shah and Mohit Sharma and Kathryn Shea and Charles Shu and Vikas Sindhwani and Sumeet Singh and Radu Soricut and Jost Tobias Springenberg and Rachel Sterneck and Razvan Surdulescu and Jie Tan and Jonathan Tompson and Vincent Vanhoucke and Jake Varley and Grace Vesom and Giulia Vezzani and Oriol Vinyals and Ayzaan Wahid and Stefan Welker and Paul Wohlhart and Fei Xia and Ted Xiao and Annie Xie and Jinyu Xie and Peng Xu and Sichun Xu and Ying Xu and Zhuo Xu and Yuxiang Yang and Rui Yao and Sergey Yaroshenko and Wenhao Yu and Wentao Yuan and Jingwei Zhang and Tingnan Zhang and Allan Zhou and Yuxiang Zhou},
journal= {arXiv preprint arXiv:2503.20020},
year = {2025}
}