Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Model
Abstract
Previous foundation models for fundus images were pre-trained with limited disease categories and knowledge base. Here we introduce a knowledge-rich vision-language model (RetiZero) that leverages knowledge from more than 400 fundus diseases. For RetiZero's pretraining, we compiled 341,896 fundus images paired with texts, sourced from public datasets, ophthalmic literature, and online resources, encompassing a diverse range of diseases across multiple ethnicities and countries. RetiZero exhibits remarkable performance in several downstream tasks, including zero-shot disease recognition, image-to-image retrieval, AI-assisted clinical diagnosis,few-shot fine-tuning, and internal- and cross-domain disease identification. In zero-shot scenarios, RetiZero achieves Top-5 accuracies of 0.843 for 15 diseases and 0.756 for 52 diseases. For image retrieval, it achieves Top-5 scores of 0.950 and 0.886 for the same sets, respectively. AI-assisted clinical diagnosis results show that RetiZero's Top-3 zero-shot performance surpasses the average of 19 ophthalmologists from Singapore, China, and the United States. RetiZero substantially enhances clinicians' accuracy in diagnosing fundus diseases, in particularly rare ones. These findings underscore the value of integrating the RetiZero into clinical settings, where various fundus diseases are encountered.
Keywords
Cite
@article{arxiv.2406.09317,
title = {Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Model},
author = {Meng Wang and Tian Lin and Aidi Lin and Kai Yu and Yuanyuan Peng and Lianyu Wang and Cheng Chen and Ke Zou and Huiyu Liang and Man Chen and Xue Yao and Meiqin Zhang and Binwei Huang and Chaoxin Zheng and Peixin Zhang and Wei Chen and Yilong Luo and Yifan Chen and Honghe Xia and Tingkun Shi and Qi Zhang and Jinming Guo and Xiaolin Chen and Jingcheng Wang and Yih Chung Tham and Dianbo Liu and Wendy Wong and Sahil Thakur and Beau Fenner and Danqi Fang and Siying Liu and Qingyun Liu and Yuqiang Huang and Hongqiang Zeng and Yanda Meng and Yukun Zhou and Zehua Jiang and Minghui Qiu and Changqing Zhang and Xinjian Chen and Sophia Y. Wang and Cecilia S. Lee and Lucia Sobrin and Carol Y Cheung and Chi Pui Pang and Pearse A. Keane and Ching-Yu Cheng and Haoyu Chen and Huazhu Fu},
journal= {arXiv preprint arXiv:2406.09317},
year = {2025}
}