ManiVID-3D：基于解耦 3D 表示实现机器人操作的通用视图无关强化学习
摘要
在机器人操作中部署视觉强化学习（RL）策略常因摄像机视角变化而受限。从固定前置摄像机训练的策略在摄像机移动时可能失效——这是现实环境中难以恰当安置传感器的必然情况。现有方法常依赖精确的摄像机标定，或在大幅视角变化时表现不佳。为此，我们提出了 ManiVID-3D，一种专为机器人操作设计的新型 3D RL 架构，通过自监督解耦特征学习获得视图无关表示。该框架集成了 ViewNet——一个轻量且高效的模块，可自动将来自任意视角的点云观测对齐到统一的空间坐标系，无需外参标定。此外，我们开发了高效的 GPU 加速批处理渲染模块， Capable of processing over 5000 frames per second, enabling large-scale training for 3D visual RL at unprecedented speeds. Extensive evaluation across 10 simulated and 5 real-world tasks demonstrates that our approach achieves a 40.6% higher success rate than state-of-the-art methods under viewpoint variations while using 80% fewer parameters. The system's robustness to severe perspective changes and strong sim-to-real performance highlight the effectiveness of learning geometrically consistent representations for scalable robotic manipulation in unstructured environments.
引用
@article{arxiv.2509.11125,
title = {ManiVID-3D: Generalizable View-Invariant Reinforcement Learning for Robotic Manipulation via Disentangled 3D Representations},
author = {Zheng Li and Pei Qu and Yufei Jia and Shihui Zhou and Haizhou Ge and Jiahang Cao and Jinni Zhou and Guyue Zhou and Jun Ma},
journal= {arXiv preprint arXiv:2509.11125},
year = {2026}
}
备注
Accepted to RA-L. Project website: https://zheng-joe-lee.github.io/manivid3d/