Current 3D inpainting and object removal methods are largely limited to front-facing scenes, facing substantial challenges when applied to diverse, "unconstrained" scenes where the camera orientation and trajectory are unrestricted. To bridge this gap, we introduce a novel approach that produces inpainted 3D scenes with consistent visual quality and coherent underlying geometry across both front-facing and unconstrained scenes. Specifically, we propose a robust 3D inpainting pipeline that incorporates geometric priors and a multi-view refinement network trained via test-time adaptation, building on a pre-trained image inpainting model. Additionally, we develop a novel inpainting mask detection technique to derive targeted inpainting masks from object masks, boosting the performance in handling unconstrained scenes. To validate the efficacy of our approach, we create a challenging and diverse benchmark that spans a wide range of scenes. Comprehensive experiments demonstrate that our proposed method substantially outperforms existing state-of-the-art approaches.
@article{arxiv.2503.04501,
title = {IMFine: 3D Inpainting via Geometry-guided Multi-view Refinement},
author = {Zhihao Shi and Dong Huo and Yuhongze Zhou and Kejia Yin and Yan Min and Juwei Lu and Xinxin Zuo},
journal= {arXiv preprint arXiv:2503.04501},
year = {2025}
}
Comments
Accepted at CVPR 2025, \href{https://xinxinzuo2353.github.io/imfine/}{Project Page}