Evaluating Large Language Models for Radiology Natural Language Processing
Abstract
The rise of large language models (LLMs) has marked a pivotal shift in the field of natural language processing (NLP). LLMs have revolutionized a multitude of domains, and they have made a significant impact in the medical field. Large language models are now more abundant than ever, and many of these models exhibit bilingual capabilities, proficient in both English and Chinese. However, a comprehensive evaluation of these models remains to be conducted. This lack of assessment is especially apparent within the context of radiology NLP. This study seeks to bridge this gap by critically evaluating thirty two LLMs in interpreting radiology reports, a crucial component of radiology NLP. Specifically, the ability to derive impressions from radiologic findings is assessed. The outcomes of this evaluation provide key insights into the performance, strengths, and weaknesses of these LLMs, informing their practical applications within the medical domain.
Cite
@article{arxiv.2307.13693,
title = {Evaluating Large Language Models for Radiology Natural Language Processing},
author = {Zhengliang Liu and Tianyang Zhong and Yiwei Li and Yutong Zhang and Yi Pan and Zihao Zhao and Peixin Dong and Chao Cao and Yuxiao Liu and Peng Shu and Yaonai Wei and Zihao Wu and Chong Ma and Jiaqi Wang and Sheng Wang and Mengyue Zhou and Zuowei Jiang and Chunlin Li and Jason Holmes and Shaochen Xu and Lu Zhang and Haixing Dai and Kai Zhang and Lin Zhao and Yuanhao Chen and Xu Liu and Peilong Wang and Junhao Chen and Pingkun Yan and Jun Liu and Bao Ge and Lichao Sun and Dajiang Zhu and Xiang Li and Wei Liu and Xiaoyan Cai and Xintao Hu and Xi Jiang and Shu Zhang and Xin Zhang and Tuo Zhang and Shijie Zhao and Quanzheng Li and Hongtu Zhu and Dinggang Shen and Tianming Liu},
journal= {arXiv preprint arXiv:2307.13693},
year = {2025}
}