Understanding and generating the fine-grained structure of objects -- such as birds with species-specific beaks, wings, and tails -- is a long-standing challenge in computer vision. We propose Chirpy3D, a part-aware multi-view diffusion framework that learns a hierarchical part latent space from unposed 2D images, using only off-the-shelf 2D part segmentation masks as spatial guidance -- without requiring any 3D data, camera poses, or manual part annotations. This latent space enables intuitive part-level swapping, interpolation, and zero-shot composition. A self-supervised feature consistency loss further encourages structural alignment across views, allowing coherent generation even with hybrid or unseen part combinations. Our core contribution is the controllable part-aware latent space and multi-view diffusion model. Downstream 3D generation is supported via any differentiable renderer such as NeRF but is orthogonal to the main framework, making Chirpy3D a flexible foundation for creative object generation in the absence of structured 3D data. Code is released at https://github.com/kamwoh/chirpy3d.
@article{arxiv.2501.04144,
title = {Chirpy3D: Part-Aware Multi-View Diffusion for Creative Fine-Grained Object Generation},
author = {Kam Woh Ng and Jing Yang and Jia Wei Sii and Chee Seng Chan and Jiankang Deng and Yi-Zhe Song and Tao Xiang and Xiatian Zhu},
journal= {arXiv preprint arXiv:2501.04144},
year = {2026}
}
Comments
20 pages. Code at https://github.com/kamwoh/chirpy3d