面向人类水平3D相对姿态估计: 无需训练的通用方法
摘要
人类可以轻易推断出未见过的物体的相对姿态, 无需标注或训练, 只需给定单个查询-参考图像对。这被认为是通过结合以下几点实现的: i) 来自单个图像的3D/2.5D形态感知, ii) 渲染-比较仿真, iii) 丰富的语义线索意识, 以提供(粗糙的)查询-参考对应关系。受此启发, 我们提出了一种 novel 3D generalizable relative pose estimation method, 通过 elaborating 3D/2.5D shape perception with a 2.5D shape from an RGB-D reference, fulfilling the render-and-compare paradigm with an off-the-shelf differentiable renderer, and leveraging the semantic cues from a pretrained model like DINOv2。具体而言, 我们的可微渲染器采用由RGB和语义图(由DINOv2从RGB输入获取)纹理化的2.5D可旋转网格, 在 novel rotated view 下渲染新的RGB和语义图(进行背面剔除)。精炼损失来自比较渲染的RGB和语义图与查询图, 通过反向传播通过可微渲染器的梯度来 refine 3D相对姿态。结果, \emph{our method can be readily applied to unseen objects, given only a single RGB-D reference, without labeling or training}。在 LineMOD, LM-O, and YCB-V 上进行的大量实验表明, 我们的方法在严格的 \texttt{Acc@5/10/15} 指标和具有挑战性的 cross-dataset 设置下, 显著优于最新的监督方法。
引用
@article{arxiv.2406.18453,
title = {Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference},
author = {Yuan Gao and Yajing Luo and Junhong Wang and Kui Jia and Gui-Song Xia},
journal= {arXiv preprint arXiv:2406.18453},
year = {2025}
}
备注
Accepted to IEEE TPAMI. The codes are available at https://github.com/ethanygao/training-free_generalizable_relative_pose