English

ConsistCompose: Unified Multimodal Layout Control for Image Composition

Computer Vision and Pattern Recognition 2026-03-17 v3

Abstract

Unified multimodal models that couple visual understanding with image generation have advanced rapidly, yet most systems still focus on visual grounding-aligning language with image regions-while their generative counterpart, linguistic-embedded layout-grounded generation (LELG) for layout-controllable multi-instance generation, remains underexplored and limits precise compositional control. We present ConsistCompose, a unified multimodal framework that embeds layout coordinates directly into language prompts, enabling layout-controlled multi-instance image generation from Interleaved Image-Text within a single generative interface. We further construct ConsistCompose3M, a 3.4M multi-instance generation dataset with layout and identity annotations (2.6M text-guided and 0.8M image-guided data pairs) that provides large-scale supervision for layout-conditioned generation. Within this framework, LELG is instantiated through instance-coordinate binding prompts and coordinate-aware classifier-free guidance, which translate linguistic layout cues into precise spatial control without task-specific branches. Experiments on COCO-Position and MS-Bench show that ConsistCompose substantially improves spatial accuracy over layout-controlled baselines while preserving identity fidelity and competitive general multimodal understanding, establishing a unified paradigm for layout-controllable multimodal image generation.

Keywords

Cite

@article{arxiv.2511.18333,
  title  = {ConsistCompose: Unified Multimodal Layout Control for Image Composition},
  author = {Xuanke Shi and Boxuan Li and Xiaoyang Han and Zhongang Cai and Lei Yang and Quan Wang and Dahua Lin},
  journal= {arXiv preprint arXiv:2511.18333},
  year   = {2026}
}

Comments

Accepted to CVPR 2026; 23 pages, 17 figures

R2 v1 2026-07-01T07:50:45.544Z