English
Related papers

Related papers: On Large Visual Language Models for Medical Imagin…

200 papers

Large language models (LLMs) have been effectively used for many computer vision tasks, including image classification. In this paper, we present a simple yet effective approach for zero-shot image classification using multimodal LLMs.…

Computer Vision and Pattern Recognition · Computer Science 2025-06-27 Abdelrahman Abdelhamed , Mahmoud Afifi , Alec Go

The advent of large vision-language models (LVLMs) represents a remarkable advance in the quest for artificial general intelligence. However, the model's effectiveness in both specialized and general tasks warrants further investigation.…

Computer Vision and Pattern Recognition · Computer Science 2024-10-29 Yao Jiang , Xinyu Yan , Ge-Peng Ji , Keren Fu , Meijun Sun , Huan Xiong , Deng-Ping Fan , Fahad Shahbaz Khan

Multi-modal learning has significantly advanced generative AI, especially in vision-language modeling. Innovations like GPT-4V and open-source projects such as LLaVA have enabled robust conversational agents capable of zero-shot task…

Computation and Language · Computer Science 2024-06-17 Zekai Chen , Arda Pekis , Kevin Brown

The emergence of large Vision-Language Models (VLMs) has established new baselines in image classification across multiple domains. We examine whether their multimodal reasoning can also address a challenge mastered by human experts.…

Computer Vision and Pattern Recognition · Computer Science 2025-09-11 Ombretta Strafforello , Derya Soydaner , Michiel Willems , Anne-Sofie Maerten , Stefanie De Winter

This paper explores training medical vision-language models (VLMs) -- where the visual and language inputs are embedded into a common space -- with a particular focus on scenarios where training data is limited, as is often the case in…

Computer Vision and Pattern Recognition · Computer Science 2023-04-03 Rhydian Windsor , Amir Jamaludin , Timor Kadir , Andrew Zisserman

Vision-Language Models (VLMs) have rapidly advanced alongside Large Language Models (LLMs). This study evaluates the capabilities of prominent generative VLMs, such as GPT-4.1 and Gemini 2.5 Pro, accessed via APIs, for histopathology image…

Computer Vision and Pattern Recognition · Computer Science 2025-06-17 Samarth Singhal , Sandeep Singhal

Large Language Models (LLMs) are increasingly applied to medical imaging tasks, including image interpretation and synthetic image generation. However, these models often produce hallucinations, which are confident but incorrect outputs…

Image and Video Processing · Electrical Eng. & Systems 2025-08-12 Anindya Bijoy Das , Shahnewaz Karim Sakib , Shibbir Ahmed

The adoption of large language models (LLMs) to assist clinicians has attracted remarkable attention. Existing works mainly adopt the close-ended question-answering (QA) task with answer options for evaluation. However, many clinical…

Vision language models (VLMs) have recently emerged and gained the spotlight for their ability to comprehend the dual modality of image and textual data. VLMs such as LLaVA, ChatGPT-4, and Gemini have recently shown impressive performance…

Computer Vision and Pattern Recognition · Computer Science 2024-05-03 Prateek Verma , Minh-Hao Van , Xintao Wu

Surgery is a highly complex process, and artificial intelligence has emerged as a transformative force in supporting surgical guidance and decision-making. However, the unimodal nature of most current AI systems limits their ability to…

Computer Vision and Pattern Recognition · Computer Science 2026-01-26 Nakul Poudel , Richard Simon , Cristian A. Linte

The development of vision-language models (VLMs) for histo-pathology has shown promising new usages and zero-shot performances. However, current approaches, which decompose large slides into smaller patches, focus solely on inductive…

Computer Vision and Pattern Recognition · Computer Science 2024-09-04 Maxime Zanella , Fereshteh Shakeri , Yunshi Huang , Houda Bahig , Ismail Ben Ayed

Large language models (LLMs) are increasingly used to extract structured information from free-text clinical records, but prior work often focuses on single tasks, limited models, and English-language reports. We evaluated 15 open-weight…

Recent Vision Language Models (VLMs) have demonstrated strong performance across a wide range of multimodal reasoning tasks. This raises the question of whether such general-purpose models can also address specialized visual recognition…

Computer Vision and Pattern Recognition · Computer Science 2026-04-14 Vaclav Javorek , Jakub Honzik , Ivan Gruber , Tomas Zelezny , Marek Hruz

The application of large language models (LLMs) in healthcare has gained significant attention due to their ability to process complex medical data and provide insights for clinical decision-making. These models have demonstrated…

Computer Vision and Pattern Recognition · Computer Science 2024-09-26 Amna Khalid , Ayma Khalid , Umar Khalid

Large Language Models (LLMs) have introduced a new era of proficiency in comprehending complex healthcare and biomedical topics. However, there is a noticeable lack of models in languages other than English and models that can interpret…

Computer Vision and Pattern Recognition · Computer Science 2023-11-02 Junling Liu , Ziming Wang , Qichen Ye , Dading Chong , Peilin Zhou , Yining Hua

Visual Language Models (VLMs) are now increasingly being merged with Large Language Models (LLMs) to enable new capabilities, particularly in terms of improved interactivity and open-ended responsiveness. While these are remarkable…

The deployment of large language models (LLMs) within the healthcare sector has sparked both enthusiasm and apprehension. These models exhibit the remarkable capability to provide proficient responses to free-text queries, demonstrating a…

Computation and Language · Computer Science 2024-07-09 Zabir Al Nazi , Wei Peng

Large Language Models (LLMs) have rapidly become important tools in Biomedical and Health Informatics (BHI), enabling new ways to analyze data, treat patients, and conduct research. This study aims to provide a comprehensive overview of LLM…

Digital Libraries · Computer Science 2024-07-30 Huizi Yu , Lizhou Fan , Lingyao Li , Jiayan Zhou , Zihui Ma , Lu Xian , Wenyue Hua , Sijia He , Mingyu Jin , Yongfeng Zhang , Ashvin Gandhi , Xin Ma

Several recent works seek to develop foundation models specifically for medical applications, adapting general-purpose large language models (LLMs) and vision-language models (VLMs) via continued pretraining on publicly available biomedical…

Computation and Language · Computer Science 2024-11-21 Daniel P. Jeong , Saurabh Garg , Zachary C. Lipton , Michael Oberst

Large multimodal language models (LMMs) have achieved significant success in general domains. However, due to the significant differences between medical images and text and general web content, the performance of LMMs in medical scenarios…

Computer Vision and Pattern Recognition · Computer Science 2023-06-23 Weihao Gao , Zhuo Deng , Zhiyuan Niu , Fuju Rong , Chucheng Chen , Zheng Gong , Wenze Zhang , Daimin Xiao , Fang Li , Zhenjie Cao , Zhaoyi Ma , Wenbin Wei , Lan Ma