English
Related papers

Related papers: Simulating Clinical AI Assistance using Multimodal…

200 papers

Diabetic retinopathy (DR) is a leading cause of preventable blindness, affecting over 100 million people worldwide. In the United States, individuals from lower-income communities face a higher risk of progressing to advanced stages before…

Computer Vision and Pattern Recognition · Computer Science 2025-07-22 Jeannie She , Katie Spivakovsky

Large language models (LLMs) show promise in automating clinical diagnosis, yet their non-transparent decision-making and limited alignment with diagnostic standards hinder trust and clinical adoption. We address this challenge by proposing…

Artificial Intelligence · Computer Science 2025-11-25 Yining Yuan , J. Ben Tamo , Micky C. Nnamdi , Yifei Wang , May D. Wang

This paper proposes one of the first clinical applications of multimodal large language models (LLMs) as an assistant for radiologists to check errors in their reports. We created an evaluation dataset from real-world radiology datasets…

Computation and Language · Computer Science 2024-03-05 Jinge Wu , Yunsoo Kim , Eva C. Keller , Jamie Chow , Adam P. Levine , Nikolas Pontikos , Zina Ibrahim , Paul Taylor , Michelle C. Williams , Honghan Wu

This paper explores the effectiveness of Multimodal Large Language models (MLLMs) as assistive technologies for visually impaired individuals. We conduct a user survey to identify adoption patterns and key challenges users face with such…

The ability to predict drug overdose risk from a patient's medical records is crucial for timely intervention and prevention. Traditional machine learning models have shown promise in analyzing longitudinal medical records for this task.…

Artificial Intelligence · Computer Science 2025-04-17 Md Sultan Al Nahian , Chris Delcher , Daniel Harris , Peter Akpunonu , Ramakanth Kavuluru

Diabetic Retinopathy (DR) is a leading cause of vision loss in the world,. In the past few Diabetic Retinopathy (DR) is a leading cause of vision loss in the world. In the past few years, Artificial Intelligence (AI) based approaches have…

Image and Video Processing · Electrical Eng. & Systems 2021-07-02 Vasudevan Lakshminarayanan , Hoda Kherdfallah , Arya Sarkar , J. Jothi Balaji

Several medical Multimodal Large Languange Models (MLLMs) have been developed to address tasks involving visual images with textual instructions across various medical modalities, achieving impressive results. Most current medical…

Computer Vision and Pattern Recognition · Computer Science 2025-04-08 Lehan Wang , Haonan Wang , Honglong Yang , Jiaji Mao , Zehong Yang , Jun Shen , Xiaomeng Li

Background: The potential of large language models (LLMs) to automate and support pharmacoepidemiologic study design is an emerging area of interest, yet their reliability remains insufficiently characterized. General-purpose LLMs often…

Computation and Language · Computer Science 2026-04-21 Xinyao Zhang , Nicole Sonne Heckmann , Manuela Del Castillo Suero , Francesco Paolo Speca , Maurizio Sessa

Diabetic Retinopathy (DR) is a leading cause of vision loss in working-age individuals. Early detection of DR can reduce the risk of vision loss by up to 95%, but a shortage of retinologists and challenges in timely examination complicate…

Diabetes mellitus (DM) predisposes patients to vascular complications. Retinal images and vasculature reflect the body's micro- and macrovascular health. They can be used to diagnose DM complications, including diabetic retinopathy (DR),…

Mental health disorders remain among the leading cause of disability worldwide, yet conditions such as depression, anxiety, and Post-Traumatic Stress Disorder (PTSD) are frequently underdiagnosed or misdiagnosed due to subjective…

Computation and Language · Computer Science 2025-10-17 Jianfeng Zhu , Julina Maharjan , Xinyu Li , Karin G. Coifman , Ruoming Jin

Diabetic retinopathy (DR) is a leading cause of blindness worldwide, necessitating early detection to prevent vision loss. Current automated DR detection systems often struggle with poor-quality images, lack interpretability, and…

Image and Video Processing · Electrical Eng. & Systems 2025-09-03 Idowu Paul Okuwobi , Jingyuan Liu , Jifeng Wan , Jiaojiao Jiang

Purpose: Large language models (LLMs) have proven performance for certain diagnostic tasks, however limited studies have evaluated their consistency in recommending appropriate medication regimens for a given diagnosis. Medication…

Diabetic retinopathy (DR) is a leading cause of preventable blindness worldwide, demanding accurate automated diagnostic systems. While general-domain vision-language models like Contrastive Language-Image Pre-Training (CLIP) perform well…

Computer Vision and Pattern Recognition · Computer Science 2025-12-23 Argha Kamal Samanta , Harshika Goyal , Vasudha Joshi , Tushar Mungle , Pabitra Mitra

Large Language Models (LLMs) like GPT-4, MedPaLM-2, and Med-Gemini achieve performance competitively with human experts across various medical benchmarks. However, they still face challenges in making professional diagnoses akin to…

Computation and Language · Computer Science 2024-08-23 Xiaohan Wang , Xiaoyan Yang , Yuqi Zhu , Yue Shen , Jian Wang , Peng Wei , Lei Liang , Jinjie Gu , Huajun Chen , Ningyu Zhang

Accurate diagnosis of ocular surface diseases is critical in optometry and ophthalmology, which hinge on integrating clinical data sources (e.g., meibography imaging and clinical metadata). Traditional human assessments lack precision in…

Computation and Language · Computer Science 2024-10-02 Chun-Hsiao Yeh , Jiayun Wang , Andrew D. Graham , Andrea J. Liu , Bo Tan , Yubei Chen , Yi Ma , Meng C. Lin

The rapid development of multimodal large language models (MLLMs), such as GPT-4V, has led to significant advancements. However, these models still face challenges in medical multimodal capabilities due to limitations in the quantity and…

Computer Vision and Pattern Recognition · Computer Science 2024-10-01 Junying Chen , Chi Gui , Ruyi Ouyang , Anningzhe Gao , Shunian Chen , Guiming Hardy Chen , Xidong Wang , Ruifei Zhang , Zhenyang Cai , Ke Ji , Guangjun Yu , Xiang Wan , Benyou Wang

Diabetic retinopathy is a leading cause of blindness in diabetic patients and early detection plays a crucial role in preventing vision loss. Traditional diagnostic methods are often time-consuming and prone to errors. The emergence of deep…

Computer Vision and Pattern Recognition · Computer Science 2025-03-19 Saif Ur Rehman Khan , Muhammad Nabeel Asim , Sebastian Vollmer , Andreas Dengel

MLLMs have recently become a focal point in the field of artificial intelligence research. Building on the strong capabilities of LLMs, MLLMs are adept at addressing complex multi-modal tasks. With the release of GPT-4, MLLMs have gained…

Machine Learning · Computer Science 2025-05-01 Jiarui Ye , Hao Tang

Background: The worst outcomes of diabetic retinopathy (DR) can be prevented by implementing DR screening programs assisted by AI. At the University Hospital of Navarre (HUN), Spain, general practitioners (GPs) grade fundus images in an…

Image and Video Processing · Electrical Eng. & Systems 2024-12-20 Imanol Pinto , Álvaro Olazarán , David Jurío , Borja de la Osa , Miguel Sainz , Aritz Oscoz , Jerónimo Ballaz , Javier Gorricho , Mikel Galar , José Andonegui