English
Related papers

Related papers: VILA-M3: Enhancing Vision-Language Models with Med…

200 papers

Vision-language models (VLMs) are increasingly adapted through domain-specific fine-tuning, yet it remains unclear whether this improves reasoning beyond superficial visual cues, particularly in high-stakes domains like medicine. We…

Computer Vision and Pattern Recognition · Computer Science 2026-04-14 Oliver McLaughlin , Daniel Shubin , Carsten Eickhoff , Ritambhara Singh , William Rudman , Michal Golovanevsky

Large language models (LLMs) have demonstrated immense capabilities in understanding textual data and are increasingly being adopted to help researchers accelerate scientific discovery through knowledge extraction (information retrieval),…

Computer Vision and Pattern Recognition · Computer Science 2025-05-30 Robinson Umeike , Neil Getty , Fangfang Xia , Rick Stevens

While Large Language Models (LLMs) excel in world knowledge understanding, adapting them to specific subfields requires precise adjustments. Due to the model's vast scale, traditional global fine-tuning methods for large models can be…

Computer Vision and Pattern Recognition · Computer Science 2024-07-09 Jiawei Chen , Yue Jiang , Dingkang Yang , Mingcheng Li , Jinjie Wei , Ziyun Qian , Lihua Zhang

Vision-Language Models (VLMs) trained on web-scale corpora excel at natural image tasks and are increasingly repurposed for healthcare; however, their competence in medical tasks remains underexplored. We present a comprehensive evaluation…

Computer Vision and Pattern Recognition · Computer Science 2025-07-21 Che Liu , Jiazhen Pan , Weixiang Shen , Wenjia Bai , Daniel Rueckert , Rossella Arcucci

Several recent works seek to develop foundation models specifically for medical applications, adapting general-purpose large language models (LLMs) and vision-language models (VLMs) via continued pretraining on publicly available biomedical…

Computation and Language · Computer Science 2024-11-21 Daniel P. Jeong , Saurabh Garg , Zachary C. Lipton , Michael Oberst

Modern Vision-Language Models (VLMs) exhibit unprecedented capabilities in cross-modal semantic understanding between visual and textual modalities. Given the intrinsic need for multi-modal integration in clinical applications, VLMs have…

Image and Video Processing · Electrical Eng. & Systems 2025-06-24 Haoneng Lin , Cheng Xu , Jing Qin

A visual-language model (VLM) pre-trained on natural images and text pairs poses a significant barrier when applied to medical contexts due to domain shift. Yet, adapting or fine-tuning these VLMs for medical use presents considerable…

Computer Vision and Pattern Recognition · Computer Science 2024-05-31 Aisha Urooj Khan , John Garrett , Tyler Bradshaw , Lonie Salkowski , Jiwoong Jason Jeong , Amara Tariq , Imon Banerjee

The advent of large vision-language models (LVLMs) represents a remarkable advance in the quest for artificial general intelligence. However, the model's effectiveness in both specialized and general tasks warrants further investigation.…

Computer Vision and Pattern Recognition · Computer Science 2024-10-29 Yao Jiang , Xinyu Yan , Ge-Peng Ji , Keren Fu , Meijun Sun , Huan Xiong , Deng-Ping Fan , Fahad Shahbaz Khan

Inevitable domain and task discrepancies in real-world scenarios can impair the generalization performance of the pre-trained deep models for medical data. Therefore, we audaciously propose that we should build a general-purpose medical AI…

Computer Vision and Pattern Recognition · Computer Science 2023-03-14 Huahui Yi , Ziyuan Qin , Qicheng Lao , Wei Xu , Zekun Jiang , Dequan Wang , Shaoting Zhang , Kang Li

The emergence of multimodal large models (MLMs) has significantly advanced the field of visual understanding, offering remarkable capabilities in the realm of visual question answering (VQA). Yet, the true challenge lies in the domain of…

Computation and Language · Computer Science 2024-08-27 Yunxin Li , Longyue Wang , Baotian Hu , Xinyu Chen , Wanqi Zhong , Chenyang Lyu , Wei Wang , Min Zhang

Recent advancements in general-purpose or domain-specific multimodal large language models (LLMs) have witnessed remarkable progress for medical decision-making. However, they are designated for specific classification or generative tasks,…

Computer Vision and Pattern Recognition · Computer Science 2024-09-04 Songtao Jiang , Tuo Zheng , Yan Zhang , Yeying Jin , Li Yuan , Zuozhu Liu

Recent research looks to harness the general knowledge and reasoning of large language models (LLMs) into agents that accomplish user-specified goals in interactive environments. Vision-language models (VLMs) extend LLMs to multi-modal data…

Machine Learning · Computer Science 2025-05-07 Jake Grigsby , Yuke Zhu , Michael Ryoo , Juan Carlos Niebles

Medical vision-and-language pre-training (Med-VLP) has received considerable attention owing to its applicability to extracting generic vision-and-language representations from medical images and texts. Most existing methods mainly contain…

Computation and Language · Computer Science 2022-09-16 Zhihong Chen , Guanbin Li , Xiang Wan

Open-source, multilingual medical large language models (LLMs) have the potential to serve linguistically diverse populations across different regions. Adapting generic LLMs for healthcare often requires continual pretraining, but this…

Computation and Language · Computer Science 2024-09-10 Meng Zhou , Surajsinh Parmar , Anubhav Bhatti

Frontier large language models (LLMs), such as GPT-5, Claude 4.5, Gemini 3, Llama 4, and DeepSeek-R1, represent a transformative class of AI tools capable of revolutionizing various aspects of healthcare by generating human-like responses…

Visual Question-Answering (VQA) has become key to user experience, particularly after improved generalization capabilities of Vision-Language Models (VLMs). But evaluating VLMs for an application requirement using a standardized framework…

Computer Vision and Pattern Recognition · Computer Science 2024-12-13 Neelabh Sinha , Vinija Jain , Aman Chadha

The large-scale pre-trained vision language models (VLM) have shown remarkable domain transfer capability on natural images. However, it remains unknown whether this capability can also apply to the medical image domain. This paper…

Computer Vision and Pattern Recognition · Computer Science 2023-02-08 Ziyuan Qin , Huahui Yi , Qicheng Lao , Kang Li

Vision Language Models (VLMs) have shown promise in automating image diagnosis and interpretation in clinical settings. However, developing specialist medical VLMs requires substantial computational resources and carefully curated datasets,…

Image and Video Processing · Electrical Eng. & Systems 2026-03-31 Yuan Zhong , Ruinan Jin , Qi Dou , Xiaoxiao Li

Vision-language models (VLMs) have made substantial progress across a wide range of visual question answering benchmarks, spanning visual reasoning, document understanding, and multimodal dialogue. These improvements are evident in a wide…

Computer Vision and Pattern Recognition · Computer Science 2026-02-23 Dhruba Ghosh , Yuhui Zhang , Ludwig Schmidt

Large Vision-Language Models offer a new paradigm for AI-driven image understanding, enabling models to perform tasks without task-specific training. This flexibility holds particular promise across medicine, where expert-annotated data is…

Computer Vision and Pattern Recognition · Computer Science 2025-04-04 Anita Rau , Mark Endo , Josiah Aklilu , Jaewoo Heo , Khaled Saab , Alberto Paderno , Jeffrey Jopling , F. Christopher Holsinger , Serena Yeung-Levy
‹ Prev 1 2 3 10 Next ›