English
Related papers

Related papers: Evaluating Large Language Models on Rare Disease D…

200 papers

Artificially intelligent (AI) co-scientists must be able to sift through research literature cost-efficiently while applying nuanced scientific reasoning. We evaluate Small Language Models (SLMs, <= 8B parameters) for classifying medical…

Computational Engineering, Finance, and Science · Computer Science 2025-12-09 Muhammed Muaaz Dawood , Mohammad Zaid Moonsamy , Kaela Kokkas , Hairong Wang , Robert F. Breiman , Richard Klein , Emmanuel K. Sekyi , Bruce A. Bassett

In health, most large language model (LLM) research has focused on clinical tasks. However, mobile and wearable devices, which are rarely integrated into such tasks, provide rich, longitudinal data for personal health monitoring. Here we…

Large language models (LLMs) have shown potential in biomedical applications, leading to efforts to fine-tune them on domain-specific data. However, the effectiveness of this approach remains unclear. This study evaluates the performance of…

Large Language Models (LLMs) have demonstrated promise in medical knowledge assessments, yet their practical utility in real-world clinical decision-making remains underexplored. In this study, we evaluated the performance of three…

Computation and Language · Computer Science 2025-12-30 Mengdi Chai , Ali R. Zomorrodi

Large language models (LLMs) are emerging as promising tools for mental health care, offering scalable support through their ability to generate human-like responses. However, the effectiveness of these models in clinical settings remains…

Artificial Intelligence · Computer Science 2024-08-22 Yining Hua , Hongbin Na , Zehan Li , Fenglin Liu , Xiao Fang , David Clifton , John Torous

Recently, large language models (LLMs) have outperformed human experts in predicting the results of neuroscience experiments (Luo et al., 2024). What is the basis for this performance? One possibility is that statistical patterns in that…

Neurons and Cognition · Quantitative Biology 2024-07-03 Xiaoliang Luo , Guangzhi Sun , Bradley C. Love

Understanding the conversation abilities of Large Language Models (LLMs) can help lead to its more cautious and appropriate deployment. This is especially important for safety-critical domains like mental health, where someone's life may…

Computation and Language · Computer Science 2024-03-18 Alexander Marrapese , Basem Suleiman , Imdad Ullah , Juno Kim

This study aims to simulate real-world clinical scenarios to systematically evaluate the ability of Large Language Models (LLMs) to extract core medical information from patient chief complaints laden with noise and redundancy, and to…

Artificial Intelligence · Computer Science 2025-12-15 Yuan Shen , Xiaojun Wu , Linghua Yu

Large language models (LLMs) have shown impressive capabilities in natural language processing tasks, including dialogue generation. This research aims to conduct a novel comparative analysis of two prominent techniques, fine-tuning with…

Medical benchmark datasets significantly contribute to developing Large Language Models (LLMs) for medical knowledge extraction, diagnosis, summarization, and other uses. Yet, current benchmarks are mainly derived from exam questions given…

Computation and Language · Computer Science 2025-03-11 Oriel Perets , Ofir Ben Shoham , Nir Grinberg , Nadav Rappoport

Recent advances in reasoning with large language models (LLMs)has shown remarkable reasoning capabilities in domains such as mathematics and coding, yet their application to clinical diagnosis remains underexplored. Here, we introduce…

Computation and Language · Computer Science 2025-04-16 Wuyang Lan , Wenzheng Wang , Changwei Ji , Guoxing Yang , Yongbo Zhang , Xiaohong Liu , Song Wu , Guangyu Wang

Recent advances in Large Language Models (LLMs) have presented new opportunities for integrating Artificial General Intelligence (AGI) into biological research and education. This study evaluated the capabilities of leading LLMs, including…

Computation and Language · Computer Science 2023-11-15 Xinyu Gong , Jason Holmes , Yiwei Li , Zhengliang Liu , Qi Gan , Zihao Wu , Jianli Zhang , Yusong Zou , Yuxi Teng , Tian Jiang , Hongtu Zhu , Wei Liu , Tianming Liu , Yajun Yan

The current work investigates the capability of Large language models (LLMs) that are explicitly trained on large corpuses of medical knowledge (Med-PaLM 2) to predict psychiatric functioning from patient interviews and clinical…

Computation and Language · Computer Science 2023-08-04 Isaac R. Galatzer-Levy , Daniel McDuff , Vivek Natarajan , Alan Karthikesalingam , Matteo Malgaroli

The deployment of Large Language Models (LLM) on mobile devices offers significant potential for medical applications, enhancing privacy, security, and cost-efficiency by eliminating reliance on cloud-based services and keeping sensitive…

Computation and Language · Computer Science 2025-02-14 Leon Nissen , Philipp Zagar , Vishnu Ravi , Aydin Zahedivash , Lara Marie Reimer , Stephan Jonas , Oliver Aalami , Paul Schmiedmayer

Large language models (LLMs) hold transformative potential in healthcare, yet their capacity to interpret longitudinal patient narratives remains inadequately explored. Dentistry, with its rich repository of structured clinical data,…

Computation and Language · Computer Science 2025-09-03 Hexian Zhang , Xinyu Yan , Yanqi Yang , Lijian Jin , Ping Yang , Junwen Wang

There is enormous enthusiasm and concerns in using large language models (LLMs) in healthcare, yet current assumptions are all based on general-purpose LLMs such as ChatGPT. This study develops a clinical generative LLM, GatorTronGPT, using…

With the significant expansion of the context window in Large Language Models (LLMs), these models are theoretically capable of processing millions of tokens in a single pass. However, research indicates a significant gap between this…

Computation and Language · Computer Science 2026-02-25 Nima Esmi , Maryam Nezhad-Moghaddam , Fatemeh Borhani , Asadollah Shahbahrami , Amin Daemdoost , Georgi Gaydadjiev

Previous research has shown that journal article quality ratings from the cloud based Large Language Model (LLM) families ChatGPT and Gemini and the medium sized open weights LLM Gemma3 27b correlate moderately with expert research quality…

Digital Libraries · Computer Science 2026-02-18 Mike Thelwall , Ehsan Mohammadi

A seminal paper published by Ledley and Lusted in 1959 introduced complex clinical diagnostic reasoning cases as the gold standard for the evaluation of expert medical computing systems, a standard that has held ever since. Here, we report…

An accurate differential diagnosis (DDx) is a cornerstone of medical care, often reached through an iterative process of interpretation that combines clinical history, physical examination, investigations and procedures. Interactive…