Large Language Models (LLMs) achieve competitive results compared to human experts in medical examinations. However, it remains a challenge to apply LLMs to complex clinical decision-making, which requires a deep understanding of medical knowledge and differs from the standardized, exam-style scenarios commonly used in current efforts. A common approach is to fine-tune LLMs for target tasks, which, however, not only requires substantial data and computational resources but also remains prone to generating `hallucinations'. In this work, we present RiskAgent, which synergizes language models with hundreds of validated clinical decision tools supported by evidence-based medicine, to provide generalizable and faithful recommendations. Our experiments show that RiskAgent not only achieves superior performance on a broad range of clinical risk predictions across diverse scenarios and diseases, but also demonstrates robust generalization in tool learning on the external MedCalc-Bench dataset, as well as in medical reasoning and question answering on three representative benchmarks, MedQA, MedMCQA, and MMLU.
@article{arxiv.2503.03802,
title = {RiskAgent: Synergizing Language Models with Validated Tools for Evidence-Based Risk Prediction},
author = {Fenglin Liu and Jinge Wu and Hongjian Zhou and Xiao Gu and Jiayuan Zhu and Jiazhen Pan and Junde Wu and Soheila Molaei and Anshul Thakur and Lei Clifton and Honghan Wu and David A. Clifton},
journal= {arXiv preprint arXiv:2503.03802},
year = {2026}
}
Comments
Code and Data are available at https://github.com/AI-in-Health/RiskAgent