In this paper, we study the problem of continuous 3D shape representations. The majority of existing successful methods are coordinate-based implicit neural representations. However, they are inefficient to render novel views or recover explicit surface points. A few works start to formulate 3D shapes as ray-based neural functions, but the learned structures are inferior due to the lack of multi-view geometry consistency. To tackle these challenges, we propose a new framework called RayDF. It consists of three major components: 1) the simple ray-surface distance field, 2) the novel dual-ray visibility classifier, and 3) a multi-view consistency optimization module to drive the learned ray-surface distances to be multi-view geometry consistent. We extensively evaluate our method on three public datasets, demonstrating remarkable performance in 3D surface point reconstruction on both synthetic and challenging real-world 3D scenes, clearly surpassing existing coordinate-based and ray-based baselines. Most notably, our method achieves a 1000x faster speed than coordinate-based methods to render an 800x800 depth image, showing the superiority of our method for 3D shape representation. Our code and data are available at https://github.com/vLAR-group/RayDF
@article{arxiv.2310.19629,
title = {RayDF: Neural Ray-surface Distance Fields with Multi-view Consistency},
author = {Zhuoman Liu and Bo Yang and Yan Luximon and Ajay Kumar and Jinxi Li},
journal= {arXiv preprint arXiv:2310.19629},
year = {2023}
}
Comments
Added the last 3 authors in the camera-ready version. NeurIPS 2023. Code and data are available at: https://github.com/vLAR-group/RayDF