English

KG-CMI: Knowledge graph enhanced cross-Mamba interaction for medical visual question answering

Computer Vision and Pattern Recognition 2026-04-02 v1

Abstract

Medical visual question answering (Med-VQA) is a crucial multimodal task in clinical decision support and telemedicine. Recent methods fail to fully leverage domain-specific medical knowledge, making it difficult to accurately associate lesion features in medical images with key diagnostic criteria. Additionally, classification-based approaches typically rely on predefined answer sets. Treating Med-VQA as a simple classification problem limits its ability to adapt to the diversity of free-form answers and may overlook detailed semantic information in those answers. To address these challenges, we propose a knowledge graph enhanced cross-Mamba interaction (KG-CMI) framework, which consists of a fine-grained cross-modal feature alignment (FCFA) module, a knowledge graph embedding (KGE) module, a cross-modal interaction representation (CMIR) module, and a free-form answer enhanced multi-task learning (FAMT) module. The KG-CMI learns cross-modal feature representations for images and texts by effectively integrating professional medical knowledge through a graph, establishing associations between lesion features and disease knowledge. Moreover, FAMT leverages auxiliary knowledge from open-ended questions, improving the model's capability for open-ended Med-VQA. Experimental results demonstrate that KG-CMI outperforms existing state-of-the-art methods on three Med-VQA datasets, i.e., VQA-RAD, SLAKE, and OVQA. Additionally, we conduct interpretability experiments to further validate the framework's effectiveness.

Keywords

Cite

@article{arxiv.2604.00601,
  title  = {KG-CMI: Knowledge graph enhanced cross-Mamba interaction for medical visual question answering},
  author = {Xianyao Zheng and Hong Yu and Hui Cui and Changming Sun and Xiangyu Li and Ran Su and Leyi Wei and Jia Zhou and Junbo Wang and Qiangguo Jin},
  journal= {arXiv preprint arXiv:2604.00601},
  year   = {2026}
}
R2 v1 2026-07-01T11:47:48.504Z