Neural Radiance Fields (NeRF) have emerged as a potent paradigm for representing scenes and synthesizing photo-realistic images. A main limitation of conventional NeRFs is that they often fail to produce high-quality renderings under novel viewpoints that are significantly different from the training viewpoints. In this paper, instead of exploiting few-shot image synthesis, we study the novel view extrapolation setting that (1) the training images can well describe an object, and (2) there is a notable discrepancy between the training and test viewpoints' distributions. We present RapNeRF (RAy Priors) as a solution. Our insight is that the inherent appearances of a 3D surface's arbitrary visible projections should be consistent. We thus propose a random ray casting policy that allows training unseen views using seen views. Furthermore, we show that a ray atlas pre-computed from the observed rays' viewing directions could further enhance the rendering quality for extrapolated views. A main limitation is that RapNeRF would remove the strong view-dependent effects because it leverages the multi-view consistency property.
@article{arxiv.2205.05922,
title = {Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolation},
author = {Jian Zhang and Yuanqing Zhang and Huan Fu and Xiaowei Zhou and Bowen Cai and Jinchi Huang and Rongfei Jia and Binqiang Zhao and Xing Tang},
journal= {arXiv preprint arXiv:2205.05922},
year = {2022}
}