We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously. In contrast to existing text-to-3D methods that often entail time-consuming per-prompt optimization and commonly output representations other than polygonal meshes, AToM directly generates high-quality textured meshes in less than 1 second with around 10 times reduction in the training cost, and generalizes to unseen prompts. Our key idea is a novel triplane-based text-to-mesh architecture with a two-stage amortized optimization strategy that ensures stable training and enables scalability. Through extensive experiments on various prompt benchmarks, AToM significantly outperforms state-of-the-art amortized approaches with over 4 times higher accuracy (in DF415 dataset) and produces more distinguishable and higher-quality 3D outputs. AToM demonstrates strong generalizability, offering finegrained 3D assets for unseen interpolated prompts without further optimization during inference, unlike per-prompt solutions.
@article{arxiv.2402.00867,
title = {AToM: Amortized Text-to-Mesh using 2D Diffusion},
author = {Guocheng Qian and Junli Cao and Aliaksandr Siarohin and Yash Kant and Chaoyang Wang and Michael Vasilkovsky and Hsin-Ying Lee and Yuwei Fang and Ivan Skorokhodov and Peiye Zhuang and Igor Gilitschenski and Jian Ren and Bernard Ghanem and Kfir Aberman and Sergey Tulyakov},
journal= {arXiv preprint arXiv:2402.00867},
year = {2024}
}
Comments
19 pages with appendix and references. Webpage: https://snap-research.github.io/AToM/