Achieving efficient and consistent localization a prior map remains challenging in robotics. Conventional keyframe-based approaches often suffers from sub-optimal viewpoints due to limited field of view (FOV) and/or constrained motion, thus degrading the localization performance. To address this issue, we design a real-time tightly-coupled Neural Radiance Fields (NeRF)-aided visual-inertial navigation system (VINS). In particular, by effectively leveraging the NeRF's potential to synthesize novel views, the proposed NeRF-VINS overcomes the limitations of traditional keyframe-based maps (with limited views) and optimally fuses IMU, monocular images, and synthetically rendered images within an efficient filter-based framework. This tightly-coupled fusion enables efficient 3D motion tracking with bounded errors. We extensively compare the proposed NeRF-VINS against the state-of-the-art methods that use prior map information and demonstrate its ability to perform real-time localization, at over 10 Hz, on a resource-constrained Jetson AGX Orin embedded platform.
@article{arxiv.2309.09295,
title = {NeRF-VINS: A Real-time Neural Radiance Field Map-based Visual-Inertial Navigation System},
author = {Saimouli Katragadda and Woosik Lee and Yuxiang Peng and Patrick Geneva and Chuchu Chen and Chao Guo and Mingyang Li and Guoquan Huang},
journal= {arXiv preprint arXiv:2309.09295},
year = {2024}
}