English

MIMO-NeRF: Fast Neural Rendering with Multi-input Multi-output Neural Radiance Fields

Computer Vision and Pattern Recognition 2023-10-04 v1 Artificial Intelligence Graphics Machine Learning Image and Video Processing

Abstract

Neural radiance fields (NeRFs) have shown impressive results for novel view synthesis. However, they depend on the repetitive use of a single-input single-output multilayer perceptron (SISO MLP) that maps 3D coordinates and view direction to the color and volume density in a sample-wise manner, which slows the rendering. We propose a multi-input multi-output NeRF (MIMO-NeRF) that reduces the number of MLPs running by replacing the SISO MLP with a MIMO MLP and conducting mappings in a group-wise manner. One notable challenge with this approach is that the color and volume density of each point can differ according to a choice of input coordinates in a group, which can lead to some notable ambiguity. We also propose a self-supervised learning method that regularizes the MIMO MLP with multiple fast reformulated MLPs to alleviate this ambiguity without using pretrained models. The results of a comprehensive experimental evaluation including comparative and ablation studies are presented to show that MIMO-NeRF obtains a good trade-off between speed and quality with a reasonable training time. We then demonstrate that MIMO-NeRF is compatible with and complementary to previous advancements in NeRFs by applying it to two representative fast NeRFs, i.e., a NeRF with sample reduction (DONeRF) and a NeRF with alternative representations (TensoRF).

Keywords

Cite

@article{arxiv.2310.01821,
  title  = {MIMO-NeRF: Fast Neural Rendering with Multi-input Multi-output Neural Radiance Fields},
  author = {Takuhiro Kaneko},
  journal= {arXiv preprint arXiv:2310.01821},
  year   = {2023}
}

Comments

Accepted to ICCV 2023. Project page: https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/mimo-nerf/

R2 v1 2026-06-28T12:39:08.354Z