Recent image-to-3D reconstruction models have greatly advanced geometry generation, but they still struggle to faithfully generate realistic appearance. To address this, we introduce ARM, a novel method that reconstructs high-quality 3D meshes and realistic appearance from sparse-view images. The core of ARM lies in decoupling geometry from appearance, processing appearance within the UV texture space. Unlike previous methods, ARM improves texture quality by explicitly back-projecting measurements onto the texture map and processing them in a UV space module with a global receptive field. To resolve ambiguities between material and illumination in input images, ARM introduces a material prior that encodes semantic appearance information, enhancing the robustness of appearance decomposition. Trained on just 8 H100 GPUs, ARM outperforms existing methods both quantitatively and qualitatively.
@article{arxiv.2411.10825,
title = {ARM: Appearance Reconstruction Model for Relightable 3D Generation},
author = {Xiang Feng and Chang Yu and Zoubin Bi and Yintong Shang and Feng Gao and Hongzhi Wu and Kun Zhou and Chenfanfu Jiang and Yin Yang},
journal= {arXiv preprint arXiv:2411.10825},
year = {2024}
}