English
Related papers

Related papers: Vision Language Models in Medicine

200 papers

In medical image analysis, the expertise scarcity and the high cost of data annotation limits the development of large artificial intelligence models. This paper investigates the potential of transfer learning with pre-trained…

Computer Vision and Pattern Recognition · Computer Science 2024-05-27 Jiajin Zhang , Ge Wang , Mannudeep K. Kalra , Pingkun Yan

The integration of Large Language Models (LLMs) into medical applications has sparked widespread interest across the healthcare industry, from drug discovery and development to clinical decision support, assisting telemedicine, medical…

Computation and Language · Computer Science 2024-12-03 Zifeng Wang , Hanyin Wang , Benjamin Danek , Ying Li , Christina Mack , Hoifung Poon , Yajuan Wang , Pranav Rajpurkar , Jimeng Sun

The utilization of large language models (LLMs) in the Healthcare domain has generated both excitement and concern due to their ability to effectively respond to freetext queries with certain professional knowledge. This survey outlines the…

Computation and Language · Computer Science 2025-01-28 Kai He , Rui Mao , Qika Lin , Yucheng Ruan , Xiang Lan , Mengling Feng , Erik Cambria

We aim to present a comprehensive overview of the latest advancements in utilizing Large Language Models (LLMs) within the healthcare sector, emphasizing their transformative impact across various medical domains. LLMs have become pivotal…

Computation and Language · Computer Science 2026-04-07 Ummara Mumtaz , Awais Ahmed , Summaya Mumtaz

Since the release of ChatGPT and GPT-4, large language models (LLMs) and multimodal large language models (MLLMs) have attracted widespread attention for their exceptional capabilities in understanding, reasoning, and generation,…

Computation and Language · Computer Science 2024-12-31 Hanguang Xiao , Feizhong Zhou , Xingyue Liu , Tianqi Liu , Zhipeng Li , Xin Liu , Xiaoxuan Huang

Following the recent popularity of Large Language Models (LLMs), several attempts have been made to extend them to the visual domain. From having a visual assistant that could guide us through unfamiliar environments to generative models…

The applications of Vision-Language Models (VLMs) in the field of Autonomous Driving (AD) have attracted widespread attention due to their outstanding performance and the ability to leverage Large Language Models (LLMs). By incorporating…

Computer Vision and Pattern Recognition · Computer Science 2024-06-25 Xingcheng Zhou , Mingyu Liu , Ekim Yurtsever , Bare Luka Zagar , Walter Zimmer , Hu Cao , Alois C. Knoll

Digital health tools have the potential to significantly improve the delivery of healthcare services. However, their adoption remains comparatively limited due, in part, to challenges surrounding usability and trust. Large Language Models…

Computation and Language · Computer Science 2024-03-01 Fergus Imrie , Paulius Rauba , Mihaela van der Schaar

The proliferation of Large Language Models (LLMs) in medicine has enabled impressive capabilities, yet a critical gap remains in their ability to perform systematic, transparent, and verifiable reasoning, a cornerstone of clinical practice.…

Computation and Language · Computer Science 2025-08-04 Wenxuan Wang , Zizhan Ma , Meidan Ding , Shiyi Zheng , Shengyuan Liu , Jie Liu , Jiaming Ji , Wenting Chen , Xiang Li , Linlin Shen , Yixuan Yuan

Multimodal Vision Language Models (VLMs) have emerged as a transformative topic at the intersection of computer vision and natural language processing, enabling machines to perceive and reason about the world through both visual and textual…

Computer Vision and Pattern Recognition · Computer Science 2025-04-08 Zongxia Li , Xiyang Wu , Hongyang Du , Fuxiao Liu , Huy Nghiem , Guangyao Shi

Recent advancements in Vision Language Models (VLMs) have demonstrated remarkable promise in generating visually grounded responses. However, their application in the medical domain is hindered by unique challenges. For instance, most VLMs…

Computer Vision and Pattern Recognition · Computer Science 2025-02-19 Lingxiao Luo , Bingda Tang , Xuanzhong Chen , Rong Han , Ting Chen

Large language models (LLMs) have emerged as powerful tools with transformative potential across numerous domains, including healthcare and medicine. In the medical domain, LLMs hold promise for tasks ranging from clinical decision support…

Computation and Language · Computer Science 2024-05-14 Xiaolan Chen , Jiayang Xiang , Shanfu Lu , Yexin Liu , Mingguang He , Danli Shi

Large Language Models (LLMs) have introduced a new era of proficiency in comprehending complex healthcare and biomedical topics. However, there is a noticeable lack of models in languages other than English and models that can interpret…

Computer Vision and Pattern Recognition · Computer Science 2023-11-02 Junling Liu , Ziming Wang , Qichen Ye , Dading Chong , Peilin Zhou , Yining Hua

Medical Visual Language Models have shown great potential in various healthcare applications, including medical image captioning and diagnostic assistance. However, most existing models rely on text-based instructions, limiting their…

Computer Vision and Pattern Recognition · Computer Science 2025-04-16 Tan-Hanh Pham , Chris Ngo , Trong-Duong Bui , Minh Luu Quang , Tan-Huong Pham , Truong-Son Hy

The rapid development of Artificial Intelligence (AI) has revolutionized numerous fields, with large language models (LLMs) and computer vision (CV) systems driving advancements in natural language understanding and visual processing,…

Computation and Language · Computer Science 2024-12-04 Yunkai Dang , Kaichen Huang , Jiahao Huo , Yibo Yan , Sirui Huang , Dongrui Liu , Mengxi Gao , Jie Zhang , Chen Qian , Kun Wang , Yong Liu , Jing Shao , Hui Xiong , Xuming Hu

Large Language Models (LLMs) have rapidly evolved from text-based systems to multimodal platforms, significantly impacting various sectors including healthcare. This comprehensive review explores the progression of LLMs to Multimodal Large…

Medical Vision Language Pretraining (VLP) has recently emerged as a promising solution to the scarcity of labeled data in the medical domain. By leveraging paired/unpaired vision and text datasets through self-supervised learning, models…

Computer Vision and Pattern Recognition · Computer Science 2023-12-12 Prashant Shrestha , Sanskar Amgain , Bidur Khanal , Cristian A. Linte , Binod Bhattarai

Several recent works seek to develop foundation models specifically for medical applications, adapting general-purpose large language models (LLMs) and vision-language models (VLMs) via continued pretraining on publicly available biomedical…

Computation and Language · Computer Science 2024-11-21 Daniel P. Jeong , Saurabh Garg , Zachary C. Lipton , Michael Oberst

Accurate diagnosis of ophthalmic diseases relies heavily on the interpretation of multimodal ophthalmic images, a process often time-consuming and expertise-dependent. Visual Question Answering (VQA) presents a potential interdisciplinary…

Image and Video Processing · Electrical Eng. & Systems 2024-10-23 Xiaolan Chen , Ruoyu Chen , Pusheng Xu , Weiyi Zhang , Xianwen Shang , Mingguang He , Danli Shi

The rise of Multimodal Large Language Models (MLLMs) has become a transformative force in the field of artificial intelligence, enabling machines to process and generate content across multiple modalities, such as text, images, audio, and…

Computation and Language · Computer Science 2025-12-09 Ming Li , Keyu Chen , Ziqian Bi , Ming Liu , Xinyuan Song , Zekun Jiang , Tianyang Wang , Benji Peng , Qian Niu , Junyu Liu , Jinlang Wang , Sen Zhang , Xuanhe Pan , Jiawei Xu , Pohsun Feng