English

JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation

Graphics 2026-05-21 v2 Artificial Intelligence Computation and Language Computer Vision and Pattern Recognition Machine Learning

Abstract

We present JoyAI-Image, a unified multimodal foundation model for visual understanding, text-to-image generation, and instruction-guided image editing. JoyAI-Image couples a spatially enhanced Multimodal Large Language Model (MLLM) with a Multimodal Diffusion Transformer (MMDiT), allowing perception and generation to interact through a shared multimodal interface. Around this architecture, we build a scalable training recipe that combines unified instruction tuning, long-text rendering supervision, spatially grounded data, and both general and spatial editing signals. This design gives the model broad multimodal capability while strengthening geometry-aware reasoning and controllable visual synthesis. Experiments across understanding, generation, long-text rendering, and editing benchmarks show that JoyAI-Image achieves state-of-the-art or highly competitive performance. More importantly, the bidirectional loop between enhanced understanding, controllable spatial editing, and novel-view-assisted reasoning enables the model to move beyond general visual competence toward stronger spatial intelligence. These results suggest a promising path for unified visual models in downstream applications such as vision-language-action systems and world models.

Keywords

Cite

@article{arxiv.2605.04128,
  title  = {JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation},
  author = {Lin Song and Wenbo Li and Guoqing Ma and Wei Tang and Bo Wang and Yuan Zhang and Yijun Yang and Yicheng Xiao and Jianhui Liu and Yanbing Zhang and Guohui Zhang and Wenhu Zhang and Hang Xu and Nan Jiang and Xin Han and Haoze Sun and Maoquan Zhang and Haoyang Huang and Nan Duan},
  journal= {arXiv preprint arXiv:2605.04128},
  year   = {2026}
}

Comments

Code: https://github.com/jd-opensource/JoyAI-Image

R2 v1 2026-07-01T12:51:33.053Z