English

Towards Efficient Neural Scene Graphs by Learning Consistency Fields

Computer Vision and Pattern Recognition 2022-10-11 v1

Abstract

Neural Radiance Fields (NeRF) achieves photo-realistic image rendering from novel views, and the Neural Scene Graphs (NSG) \cite{ost2021neural} extends it to dynamic scenes (video) with multiple objects. Nevertheless, computationally heavy ray marching for every image frame becomes a huge burden. In this paper, taking advantage of significant redundancy across adjacent frames in videos, we propose a feature-reusing framework. From the first try of naively reusing the NSG features, however, we learn that it is crucial to disentangle object-intrinsic properties consistent across frames from transient ones. Our proposed method, \textit{Consistency-Field-based NSG (CF-NSG)}, reformulates neural radiance fields to additionally consider \textit{consistency fields}. With disentangled representations, CF-NSG takes full advantage of the feature-reusing scheme and performs an extended degree of scene manipulation in a more controllable manner. We empirically verify that CF-NSG greatly improves the inference efficiency by using 85\% less queries than NSG without notable degradation in rendering quality. Code will be available at: https://github.com/ldynx/CF-NSG

Keywords

Cite

@article{arxiv.2210.04127,
  title  = {Towards Efficient Neural Scene Graphs by Learning Consistency Fields},
  author = {Yeji Song and Chaerin Kong and Seoyoung Lee and Nojun Kwak and Joonseok Lee},
  journal= {arXiv preprint arXiv:2210.04127},
  year   = {2022}
}

Comments

BMVC 2022, 22 pages

R2 v1 2026-06-28T03:04:42.669Z