Large Language Models versus Classical Machine Learning: Performance in COVID-19 Mortality Prediction Using High-Dimensional Tabular Data
Abstract
This study compared the performance of classical feature-based machine learning models (CMLs) and large language models (LLMs) in predicting COVID-19 mortality using high-dimensional tabular data from 9,134 patients across four hospitals. Seven CML models, including XGBoost and random forest (RF), were evaluated alongside eight LLMs, such as GPT-4 and Mistral-7b, which performed zero-shot classification on text-converted structured data. Additionally, Mistral- 7b was fine-tuned using the QLoRA approach. XGBoost and RF demonstrated superior performance among CMLs, achieving F1 scores of 0.87 and 0.83 for internal and external validation, respectively. GPT-4 led the LLM category with an F1 score of 0.43, while fine-tuning Mistral-7b significantly improved its recall from 1% to 79%, yielding a stable F1 score of 0.74 during external validation. Although LLMs showed moderate performance in zero-shot classification, fine-tuning substantially enhanced their effectiveness, potentially bridging the gap with CML models. However, CMLs still outperformed LLMs in handling high-dimensional tabular data tasks. This study highlights the potential of both CMLs and fine-tuned LLMs in medical predictive modeling, while emphasizing the current superiority of CMLs for structured data analysis.
Cite
@article{arxiv.2409.02136,
title = {Large Language Models versus Classical Machine Learning: Performance in COVID-19 Mortality Prediction Using High-Dimensional Tabular Data},
author = {Mohammadreza Ghaffarzadeh-Esfahani and Mahdi Ghaffarzadeh-Esfahani and Arian Salahi-Niri and Hossein Toreyhi and Zahra Atf and Amirali Mohsenzadeh-Kermani and Mahshad Sarikhani and Zohreh Tajabadi and Fatemeh Shojaeian and Mohammad Hassan Bagheri and Aydin Feyzi and Mohammadamin Tarighatpayma and Narges Gazmeh and Fateme Heydari and Hossein Afshar and Amirreza Allahgholipour and Farid Alimardani and Ameneh Salehi and Naghmeh Asadimanesh and Mohammad Amin Khalafi and Hadis Shabanipour and Ali Moradi and Sajjad Hossein Zadeh and Omid Yazdani and Romina Esbati and Moozhan Maleki and Danial Samiei Nasr and Amirali Soheili and Hossein Majlesi and Saba Shahsavan and Alireza Soheilipour and Nooshin Goudarzi and Erfan Taherifard and Hamidreza Hatamabadi and Jamil S Samaan and Thomas Savage and Ankit Sakhuja and Ali Soroush and Girish Nadkarni and Ilad Alavi Darazam and Mohamad Amin Pourhoseingholi and Seyed Amir Ahmad Safavi-Naini},
journal= {arXiv preprint arXiv:2409.02136},
year = {2026}
}
Comments
Code is available at: https://github.com/mohammad-gh009/Large-Language-Models-vs-Classical-Machine-learning and https://github.com/Sdamirsa/Tehran_COVID_Cohort. The datasets are available from the corresponding author on reasonable request (sdamirsa@ymail.com)