We present DreamBooth3D, an approach to personalize text-to-3D generative models from as few as 3-6 casually captured images of a subject. Our approach combines recent advances in personalizing text-to-image models (DreamBooth) with text-to-3D generation (DreamFusion). We find that naively combining these methods fails to yield satisfactory subject-specific 3D assets due to personalized text-to-image models overfitting to the input viewpoints of the subject. We overcome this through a 3-stage optimization strategy where we jointly leverage the 3D consistency of neural radiance fields together with the personalization capability of text-to-image models. Our method can produce high-quality, subject-specific 3D assets with text-driven modifications such as novel poses, colors and attributes that are not seen in any of the input images of the subject.
@article{arxiv.2303.13508,
title = {DreamBooth3D: Subject-Driven Text-to-3D Generation},
author = {Amit Raj and Srinivas Kaza and Ben Poole and Michael Niemeyer and Nataniel Ruiz and Ben Mildenhall and Shiran Zada and Kfir Aberman and Michael Rubinstein and Jonathan Barron and Yuanzhen Li and Varun Jampani},
journal= {arXiv preprint arXiv:2303.13508},
year = {2023}
}
Comments
Project page at https://dreambooth3d.github.io/ Video Summary at https://youtu.be/kKVDrbfvOoA