English
Related papers

Related papers: A Generative Foundation Model for Multimodal Histo…

200 papers

Immunohistochemical (IHC) biomarker prediction benefits from multi-modal data fusion analysis. However, the simultaneous acquisition of multi-modal data, such as genomic and pathological information, is often challenging due to cost or…

Computer Vision and Pattern Recognition · Computer Science 2025-08-26 Qibin Zhang , Xinyu Hao , Qiao Chen , Rui Xu , Fengyu Cong , Cheng Lu , Hongming Xu

Multimodal survival prediction, a crucial yet challenging task, demands the integration of multimodal medical data (\eg Whole Slide Images (WSIs) and Genomic Profiles) to achieve accurate prognostic modeling. Given the inherent…

Computer Vision and Pattern Recognition · Computer Science 2026-05-21 Huayi Wang , Haochao Ying , Yuyang Xu , Qiyao Zheng , jun wang , Cheng Zhang , Ying Sun , Jian Wu

Spatial Transcriptomics (ST) allows a high-resolution measurement of RNA sequence abundance by systematically connecting cell morphology depicted in Hematoxylin and Eosin (H&E) stained histology images to spatially resolved gene…

Computer Vision and Pattern Recognition · Computer Science 2025-03-21 Sichen Zhu , Yuchen Zhu , Molei Tao , Peng Qiu

Non-contact automatic deception detection remains challenging because visual and auditory deception cues often lack stable cross-subject patterns. In contrast, galvanic skin response (GSR) provides more reliable physiological cues and has…

Computer Vision and Pattern Recognition · Computer Science 2026-04-07 Peiyuan Jiang , Yao Liu , Yanglei Gan , Jiaye Yang , Lu Liu , Daibing Yao , Qiao Liu

Remarkable strides in computational pathology have been made in the task-agnostic foundation model that advances the performance of a wide array of downstream clinical tasks. Despite the promising performance, there are still several…

Rapid progress in computational pathology is increasingly driven by vision foundation models pretrained on vast histopathology datasets. While recent efforts have prioritized training on an ever-larger amount of patches, we take an…

Foundation models (FMs) have demonstrated strong performance across diverse pathology tasks. While there are similarities in the pre-training objectives of FMs, there is still limited understanding of their complementarity, redundancy in…

Computer Vision and Pattern Recognition · Computer Science 2025-12-15 Brennan Flannery , Thomas DeSilvio , Jane Nguyen , Satish E. Viswanath

Multimodal medical image fusion plays a crucial role in medical diagnosis by integrating complementary information from different modalities to enhance image readability and clinical applicability. However, existing methods mainly follow…

Computer Vision and Pattern Recognition · Computer Science 2025-05-20 Haozhe Xiang , Han Zhang , Yu Cheng , Xiongwen Quan , Wanwan Huang

Deep learning models have made significant advances in histological prediction tasks in recent years. However, for adaptation in clinical practice, their lack of robustness to varying conditions such as staining, scanner, hospital, and…

Image and Video Processing · Electrical Eng. & Systems 2025-06-23 David Jacob Drexlin , Jonas Dippel , Julius Hense , Niklas Prenißl , Grégoire Montavon , Frederick Klauschen , Klaus-Robert Müller

Generative modelling and synthetic data can be a surrogate for real medical imaging datasets, whose scarcity and difficulty to share can be a nuisance when delivering accurate deep learning models for healthcare applications. In recent…

Image and Video Processing · Electrical Eng. & Systems 2024-02-27 Virginia Fernandez , Walter Hugo Lopez Pinaya , Pedro Borges , Mark S. Graham , Tom Vercauteren , M. Jorge Cardoso

This study evaluates the generalisation capabilities of state-of-the-art histopathology foundation models on out-of-distribution multi-stain autoimmune Immunohistochemistry datasets. We compare 13 feature extractor models, including…

Computer Vision and Pattern Recognition · Computer Science 2024-10-30 Amaya Gallagher-Syed , Elena Pontarini , Myles J. Lewis , Michael R. Barnes , Gregory Slabaugh

Multimodal intent understanding is a significant research area that requires effective leveraging of multiple modalities to analyze human language. Existing methods face two main challenges in this domain. Firstly, they have limitations in…

Multimedia · Computer Science 2025-05-26 Hanlei Zhang , Qianrui Zhou , Hua Xu , Jianhua Su , Roberto Evans , Kai Gao

Recent advances in generative medical models are constrained by modality-specific scenarios that hinder the integration of complementary evidence from imaging, pathology, and clinical notes. This fragmentation limits their evolution into…

Computer Vision and Pattern Recognition · Computer Science 2025-10-08 Jiawei Mao , Yuhan Wang , Lifeng Chen , Can Zhao , Yucheng Tang , Dong Yang , Liangqiong Qu , Daguang Xu , Yuyin Zhou

In multi-class histopathology nuclei analysis tasks, the lack of training data becomes a main bottleneck for the performance of learning-based methods. To tackle this challenge, previous methods have utilized generative models to increase…

Computer Vision and Pattern Recognition · Computer Science 2024-09-05 Seonghui Min , Hyun-Jic Oh , Won-Ki Jeong

Virtual immunohistochemistry (IHC) staining from hematoxylin and eosin (H&E) images can accelerate diagnostics by providing preliminary molecular insight directly from routine sections, reducing the need for repeat sectioning when tissue is…

Computer Vision and Pattern Recognition · Computer Science 2026-03-16 Jillur Rahman Saurav , Thuong Le Hoai Pham , Pritam Mukherjee , Paul Yi , Brent A. Orr , Jacob M. Luber

Histopathology image analysis is fundamental to digital pathology, with hematoxylin and eosin (H&E) staining as the gold standard for diagnostic and prognostic assessments. While H&E imaging effectively highlights cellular and tissue…

Image and Video Processing · Electrical Eng. & Systems 2025-11-18 Yao Du , Jiaxin Zhuang , Xiaoyu Zheng , Jing Cong , Limei Guo , Chao He , Lin Luo , Xiaomeng Li

Background: The integration of multi-stain histopathology images through deep learning poses a significant challenge in digital histopathology. Current multi-modal approaches struggle with data heterogeneity and missing data. This study…

Computer Vision and Pattern Recognition · Computer Science 2024-09-27 Valentin Koch , Sabine Bauer , Valerio Luppberger , Michael Joner , Heribert Schunkert , Julia A. Schnabel , Moritz von Scheidt , Carsten Marr

Recently, it has become progressively more evident that classic diagnostic labels are unable to reliably describe the complexity and variability of several clinical phenotypes. This is particularly true for a broad range of neuropsychiatric…

Machine Learning · Computer Science 2024-02-28 Giovanna Maria Dimitri , Simeon Spasov , Andrea Duggento , Luca Passamonti , Pietro Li`o , Nicola Toschi

Conditional medical image generation plays an important role in many clinically relevant imaging tasks. However, existing methods still face a fundamental challenge in balancing inference efficiency, patient-specific fidelity, and…

Computer Vision and Pattern Recognition · Computer Science 2026-04-22 Zirong Li , Siyuan Mei , Weiwen Wu , Andreas Maier , Lina Gölz , Yan Xia

Generative models are increasingly used to augment medical imaging datasets for fairer AI. Yet a key assumption often goes unexamined: that generators themselves produce equally high-quality images across demographic groups. Models trained…

Computer Vision and Pattern Recognition · Computer Science 2026-03-18 Mahmoud Ibrahim , Bart Elen , Chang Sun , Gokhan Ertaylan , Michel Dumontier