We present a framework, DISORF, to enable online 3D reconstruction and visualization of scenes captured by resource-constrained mobile robots and edge devices. To address the limited computing capabilities of edge devices and potentially limited network availability, we design a framework that efficiently distributes computation between the edge device and the remote server. We leverage on-device SLAM systems to generate posed keyframes and transmit them to remote servers that can perform high-quality 3D reconstruction and visualization at runtime by leveraging recent advances in neural 3D methods. We identify a key challenge with online training where naive image sampling strategies can lead to significant degradation in rendering quality. We propose a novel shifted exponential frame sampling method that addresses this challenge for online training. We demonstrate the effectiveness of our framework in enabling high-quality real-time reconstruction and visualization of unknown scenes as they are captured and streamed from cameras in mobile robots and edge devices.
@article{arxiv.2403.00228,
title = {DISORF: A Distributed Online 3D Reconstruction Framework for Mobile Robots},
author = {Chunlin Li and Hanrui Fan and Xiaorui Huang and Ruofan Liang and Sankeerth Durvasula and Nandita Vijaykumar},
journal= {arXiv preprint arXiv:2403.00228},
year = {2024}
}