English
Related papers

Related papers: Merlin: A Computed Tomography Vision-Language Foun…

200 papers

Recent 3D CT vision-language models align volumes with reports via contrastive pretraining, but typically rely on limited public data and provide only coarse global supervision. We train a 3D CT vision-language model on 98k report-volume…

Computer Vision and Pattern Recognition · Computer Science 2026-03-03 Simon Ging , Philipp Arnold , Sebastian Walter , Hani Alnahas , Hannah Bast , Elmar Kotter , Jiancheng Yang , Behzad Bozorgtabar , Thomas Brox

The scalability of current language-image pre-training for 3D medical imaging, such as CT and MRI, is constrained by the need for radiologists to manually curate raw clinical studies. In this work, we pioneer pre-training directly on…

Computer Vision and Pattern Recognition · Computer Science 2026-02-20 Chenhui Zhao , Yiwei Lyu , Asadur Chowdury , Edward Harake , Akhil Kondepudi , Akshay Rao , Xinhai Hou , Honglak Lee , Todd Hollon

Large vision-language models (VLMs) have evolved from general-purpose applications to specialized use cases such as in the clinical domain, demonstrating potential for decision support in radiology. One promising application is assisting…

Computer Vision and Pattern Recognition · Computer Science 2026-02-25 Zhifan Jiang , Dong Yang , Vishwesh Nath , Abhijeet Parida , Nishad P. Kulkarni , Ziyue Xu , Daguang Xu , Syed Muhammad Anwar , Holger R. Roth , Marius George Linguraru

Vision language models (VLMs) can flexibly address various vision tasks through text interactions. Although successful in semantic understanding, state-of-the-art VLMs including GPT-5 still struggle in understanding 3D from 2D inputs. On…

Computer Vision and Pattern Recognition · Computer Science 2025-10-02 Zhipeng Cai , Ching-Feng Yeh , Hu Xu , Zhuang Liu , Gregory Meyer , Xinjie Lei , Changsheng Zhao , Shang-Wen Li , Vikas Chandra , Yangyang Shi

Computational anatomy allows the quantitative analysis of organs in medical images. However, most analysis is constrained to the millimeter scale because of the limited resolution of clinical computed tomography (CT). X-ray microtomography…

Computer Vision and Pattern Recognition · Computer Science 2017-02-28 Holger R. Roth , Kai Nagara , Hirohisa Oda , Masahiro Oda , Tomoshi Sugiyama , Shota Nakamura , Kensaku Mori

Medical image analysis is essential to clinical diagnosis and treatment, which is increasingly supported by multi-modal large language models (MLLMs). However, previous research has primarily focused on 2D medical images, leaving 3D images…

Computer Vision and Pattern Recognition · Computer Science 2024-04-02 Fan Bai , Yuxin Du , Tiejun Huang , Max Q. -H. Meng , Bo Zhao

Interpreting quantitative CT biomarkers, such as organ volume and tissue attenuation, requires large-scale healthy reference distributions. However, creating these is challenging because clinical datasets are often heavily enriched with…

Computer Vision and Pattern Recognition · Computer Science 2026-05-08 Christian Wachinger , Bernhard Renger , Christopher Späth , Jan Kirschke , Marcus Makowski

General-purpose large Vision-Language Models (VLMs) demonstrate strong capabilities in generating detailed descriptions for natural images. However, their performance in the medical domain remains suboptimal, even for relatively…

Computer Vision and Pattern Recognition · Computer Science 2025-10-23 Yifan Li , Fenghe Tang , Yingtai Li , Shaohua Kevin Zhou

The emergence of Large Language Models (LLMs) presents unprecedented opportunities to revolutionize medical contrastive vision-language pre-training. In this paper, we show how LLMs can facilitate large-scale supervised pre-training,…

Computer Vision and Pattern Recognition · Computer Science 2025-09-17 Yingtai Li , Haoran Lai , Xiaoqian Zhou , Shuai Ming , Wenxin Ma , Wei Wei , Shaohua Kevin Zhou

Multi-modal large language models (MLLMs) have shown promise in advancing healthcare. However, most existing models remain confined to single-image understanding, which greatly limits their applicability in clinical workflows. In practice,…

Computer Vision and Pattern Recognition · Computer Science 2025-12-01 Zhen Chen , Yihang Fu , Gabriel Madera , Mauro Giuffre , Serina Applebaum , Hyunjae Kim , Hua Xu , Qingyu Chen

The rapid advancement of artificial intelligence (AI) in healthcare imaging has revolutionized diagnostic medicine and clinical decision-making processes. This work presents an intelligent multimodal framework for medical image analysis…

Image and Video Processing · Electrical Eng. & Systems 2026-04-20 Samer Al-Hamadani

Recent advancements in Vision Language Models (VLMs) have demonstrated remarkable promise in generating visually grounded responses. However, their application in the medical domain is hindered by unique challenges. For instance, most VLMs…

Computer Vision and Pattern Recognition · Computer Science 2025-02-19 Lingxiao Luo , Bingda Tang , Xuanzhong Chen , Rong Han , Ting Chen

Current medical vision-language models (VLMs) process volumetric brain MRI using 2D slice-based approximations, fragmenting the spatial context required for accurate neuroradiological interpretation. We developed \textbf{Brain3D}, a staged…

Computer Vision and Pattern Recognition · Computer Science 2026-02-26 Mariano Barone , Francesco Di Serio , Giuseppe Riccio , Antonio Romano , Marco Postiglione , Antonino Ferraro , Vincenzo Moscato

Vision-language models (VLMs) show strong potential for complex diagnostic tasks in medical imaging. However, applying VLMs to multi-organ medical imaging introduces two principal challenges: (1) modality-specific vision-language alignment…

Computer Vision and Pattern Recognition · Computer Science 2026-03-04 Haowen Zhu , Ning Yin , Xiaogen Zhou

The advancement of artificial intelligence (AI) for organ segmentation and tumor detection is propelled by the growing availability of computed tomography (CT) datasets with detailed, per-voxel annotations. However, these AI models often…

Image and Video Processing · Electrical Eng. & Systems 2024-05-29 Jie Liu , Yixiao Zhang , Kang Wang , Mehmet Can Yavuz , Xiaoxi Chen , Yixuan Yuan , Haoliang Li , Yang Yang , Alan Yuille , Yucheng Tang , Zongwei Zhou

Vision-language models (VLMs) have demonstrated strong cross-modal capabilities, yet most work remains limited to 2D data and assumes binary supervision (i.e., positive vs. negative pairs), overlooking the continuous and structured…

Computer Vision and Pattern Recognition · Computer Science 2025-11-06 Ailar Mahdizadeh , Puria Azadi Moghadam , Xiangteng He , Shahriar Mirabbasi , Panos Nasiopoulos , Leonid Sigal

Vision-Language Pre-training (VLP) has advanced the performance of many vision-language tasks, such as image-text retrieval, visual entailment, and visual reasoning. The pre-training mostly utilizes lexical databases and image queries in…

Computation and Language · Computer Science 2023-06-30 Yasmine Karoui , Rémi Lebret , Negar Foroutan , Karl Aberer

Whole abdominal organ segmentation is important in diagnosing abdomen lesions, radiotherapy, and follow-up. However, oncologists' delineating all abdominal organs from 3D volumes is time-consuming and very expensive. Deep learning-based…

Image and Video Processing · Electrical Eng. & Systems 2023-02-14 Xiangde Luo , Wenjun Liao , Jianghong Xiao , Jieneng Chen , Tao Song , Xiaofan Zhang , Kang Li , Dimitris N. Metaxas , Guotai Wang , Shaoting Zhang

Accurate classification of focal liver lesions is crucial for diagnosis and treatment in hepatology. However, traditional supervised deep learning models depend on large-scale annotated datasets, which are often limited in medical imaging.…

Computer Vision and Pattern Recognition · Computer Science 2025-05-07 Song Jian , Hu Yuchang , Wang Hui , Chen Yen-Wei

Large Vision Language Models (LVLMs) have demonstrated remarkable capabilities, yet their proficiency in understanding and reasoning over multiple images remains largely unexplored. While existing benchmarks have initiated the evaluation of…

Computer Vision and Pattern Recognition · Computer Science 2026-01-13 Anurag Das , Adrian Bulat , Alberto Baldrati , Ioannis Maniadis Metaxas , Bernt Schiele , Georgios Tzimiropoulos , Brais Martinez