利用知识丰富的视觉语言模型提升罕见与常见眼底疾病的诊断准确性
图像与视频处理
2025-04-22 v3 计算机视觉与模式识别
摘要
以往的眼底图像基础模型是在有限的疾病类别和知识库上进行预训练的。在此，我们介绍一种知识丰富的视觉语言模型（RetiZero），该模型利用了来自400多种眼底疾病的知识。为了预训练RetiZero，我们收集了341,896张带有文本描述的眼底图像，这些图像来自公共数据集、眼科文献和在线资源，涵盖了多种族和国家的多种疾病。RetiZero在多个下游任务中表现出色，包括零样本疾病识别、图像到图像检索、AI辅助临床诊断、少样本微调以及内部和跨领域疾病识别。在零样本场景下，RetiZero在15种疾病上达到了0.843的Top-5准确率，在52种疾病上达到了0.756的Top-5准确率。在图像检索方面，它在相同疾病集上分别达到了0.950和0.886的Top-5分数。AI辅助临床诊断结果显示，RetiZero的Top-3零样本性能超过了来自新加坡、中国和美国的19名眼科医生的平均水平。RetiZero显著提高了临床医生诊断眼底疾病（尤其是罕见病）的准确性。这些发现强调了将RetiZero整合到临床环境中的价值，在该环境中会遇到各种眼底疾病。
引用
@article{arxiv.2406.09317,
title = {Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Model},
author = {Meng Wang and Tian Lin and Aidi Lin and Kai Yu and Yuanyuan Peng and Lianyu Wang and Cheng Chen and Ke Zou and Huiyu Liang and Man Chen and Xue Yao and Meiqin Zhang and Binwei Huang and Chaoxin Zheng and Peixin Zhang and Wei Chen and Yilong Luo and Yifan Chen and Honghe Xia and Tingkun Shi and Qi Zhang and Jinming Guo and Xiaolin Chen and Jingcheng Wang and Yih Chung Tham and Dianbo Liu and Wendy Wong and Sahil Thakur and Beau Fenner and Danqi Fang and Siying Liu and Qingyun Liu and Yuqiang Huang and Hongqiang Zeng and Yanda Meng and Yukun Zhou and Zehua Jiang and Minghui Qiu and Changqing Zhang and Xinjian Chen and Sophia Y. Wang and Cecilia S. Lee and Lucia Sobrin and Carol Y Cheung and Chi Pui Pang and Pearse A. Keane and Ching-Yu Cheng and Haoyu Chen and Huazhu Fu},
journal= {arXiv preprint arXiv:2406.09317},
year = {2025}
}