Customized text-to-image generation, which synthesizes images based on user-specified concepts, has made significant progress in handling individual concepts. However, when extended to multiple concepts, existing methods often struggle with properly integrating different models and avoiding the unintended blending of characteristics from distinct concepts. In this paper, we propose MC2, a novel approach for multi-concept customization that enhances flexibility and fidelity through inference-time optimization. MC2 enables the integration of multiple single-concept models with heterogeneous architectures. By adaptively refining attention weights between visual and textual tokens, our method ensures that image regions accurately correspond to their associated concepts while minimizing interference between concepts. Extensive experiments demonstrate that MC2 outperforms training-based methods in terms of prompt-reference alignment. Furthermore, MC2 can be seamlessly applied to text-to-image generation, providing robust compositional capabilities. To facilitate the evaluation of multi-concept customization, we also introduce a new benchmark, MC++. The code will be publicly available at https://github.com/JIANGJiaXiu/MC-2.
@article{arxiv.2404.05268,
title = {MC$^2$: Multi-concept Guidance for Customized Multi-concept Generation},
author = {Jiaxiu Jiang and Yabo Zhang and Kailai Feng and Xiaohe Wu and Wenbo Li and Renjing Pei and Fan Li and Wangmeng Zuo},
journal= {arXiv preprint arXiv:2404.05268},
year = {2024}
}