面向医疗的可解释视觉语言基础模型：EVLF-FM
计算机视觉与模式识别
2025-09-30 v1
摘要
尽管基础模型在医疗AI领域展现出巨大潜力，但当前系统仍受限——它们是模态特定的，缺乏透明的推理过程，阻碍了临床应用。为填补这一空白，我们提出EVLF-FM，一种面向医疗的多模态视觉语言基础模型（VLM），旨在统一广泛的诊断能力与细粒度可解释性。EVLF-FM的开发与测试涵盖超过130万个总样本，来自23个全球数据集，涉及11种成像模态，覆盖6个临床专科：皮肤科、肝胆科、眼科、病理科、肺科和放射科。外部验证采用8884个独立测试样本，来自10个额外数据集，涵盖5种成像模态。技术上，EVLF-FM旨在辅助多疾病诊断和视觉问答，具备像素级视觉锚定和推理能力。在疾病诊断的内部验证中，EVLF-FM实现了最高的平均准确率（0.858）和F1分数（0.797），超越了领先的通用和专家模型。在医学视觉锚定方面，EVLF-FM在九种模态中表现卓越，平均mIOU为0.743，[email protected]为0.837。外部验证进一步确认了强大的零样本和少样本性能，尽管模型规模较小，仍保持竞争的F1分数。通过结合监督学习和视觉强化微调的混合训练策略，EVLF-FM不仅实现了最先进的准确率，还表现出逐步推理，使输出与视觉证据保持一致。EVLF-FM是首个具备可解释性和推理能力的多疾病VLM模型，可能推动基础模型在实际临床部署中的应用与信任。
引用
@article{arxiv.2509.24231,
title = {EVLF-FM: Explainable Vision Language Foundation Model for Medicine},
author = {Yang Bai and Haoran Cheng and Yang Zhou and Jun Zhou and Arun Thirunavukarasu and Yuhe Ke and Jie Yao and Kanae Fukutsu and Chrystie Wan Ning Quek and Ashley Hong and Laura Gutierrez and Zhen Ling Teo and Darren Shu Jeng Ting and Brian T. Soetikno and Christopher S. Nielsen and Tobias Elze and Zengxiang Li and Linh Le Dinh and Hiok Hong Chan and Victor Koh and Marcus Tan and Kelvin Z. Li and Leonard Yip and Ching Yu Cheng and Yih Chung Tham and Gavin Siew Wei Tan and Leopold Schmetterer and Marcus Ang and Rahat Hussain and Jod Mehta and Tin Aung and Lionel Tim-Ee Cheng and Tran Nguyen Tuan Anh and Chee Leong Cheng and Tien Yin Wong and Nan Liu and Iain Beehuat Tan and Soon Thye Lim and Eyal Klang and Tony Kiat Hon Lim and Rick Siow Mong Goh and Yong Liu and Daniel Shu Wei Ting},
journal= {arXiv preprint arXiv:2509.24231},
year = {2025}
}