English

SynerMedGen: Synergizing Medical Multimodal Understanding with Generation via Task Alignment

Computer Vision and Pattern Recognition 2026-05-12 v1

Abstract

Unifying multimodal understanding and generation is a compelling frontier that is beginning to emerge in the medical field. However, the limited existing unified medical models typically treat understanding and generation as disjoint objectives, lacking a meaningful functional synergy. In this work, we identify and address a critical question in unified medical modeling: what form of understanding truly benefits generation. We present SynerMedGen, a unified framework built on the proposed principle of generation-aligned understanding, which synergizes understanding objectives with generation tasks via task alignment. SynerMedGen introduces three generation-aligned understanding tasks and a two-stage training strategy that transfers generation-beneficial representations learned during understanding training to medical image synthesis. Remarkably, even with understanding training alone, our SynerMedGen achieves strong zero-shot performance across 22 medical image synthesis tasks and demonstrates robust generalization to unseen datasets. When combined with generation training, SynerMedGen consistently outperforms state-of-the-art specialized medical image synthesis models as well as recent unified medical models. We also release a large-scale dataset named SynerMed consisting of 1M paired synthesis samples and 2M generation-derived understanding instances to support further research on understanding-generation synergy. Our project can be accessed at https://github.com/Mhilab/SynerMedGen.

Keywords

Cite

@article{arxiv.2605.08724,
  title  = {SynerMedGen: Synergizing Medical Multimodal Understanding with Generation via Task Alignment},
  author = {Weiren Zhao and Yi Dong and Cheng Chen},
  journal= {arXiv preprint arXiv:2605.08724},
  year   = {2026}
}

Comments

Accepted by ICML 2026

R2 v1 2026-07-01T12:59:34.451Z