We present a relocalization pipeline, which combines an absolute pose regression (APR) network with a novel view synthesis based direct matching module, offering superior accuracy while maintaining low inference time. Our contribution is twofold: i) we design a direct matching module that supplies a photometric supervision signal to refine the pose regression network via differentiable rendering; ii) we modify the rotation representation from the classical quaternion to SO(3) in pose regression, removing the need for balancing rotation and translation loss terms. As a result, our network Direct-PoseNet achieves state-of-the-art performance among all other single-image APR methods on the 7-Scenes benchmark and the LLFF dataset.
@article{arxiv.2104.04073,
title = {Direct-PoseNet: Absolute Pose Regression with Photometric Consistency},
author = {Shuai Chen and Zirui Wang and Victor Prisacariu},
journal= {arXiv preprint arXiv:2104.04073},
year = {2021}
}