English
Related papers

Related papers: Path-GPTOmic: A Balanced Multi-modal Learning Fram…

200 papers

Current multimodal survival prediction methods typically rely on pathology images (WSIs) and genomic data, both of which are high-dimensional and redundant, making it difficult to extract discriminative features from them and align…

Computer Vision and Pattern Recognition · Computer Science 2025-12-17 Chenyu Zhao , Yingxue Xu , Fengtao Zhou , Yihui Wang , Hao Chen

When oncologists estimate cancer patient survival, they rely on multimodal data. Even though some multimodal deep learning methods have been proposed in the literature, the majority rely on having two or more independent networks that share…

Image and Video Processing · Electrical Eng. & Systems 2022-09-13 Numan Saeed , Ikboljon Sobirov , Roba Al Majzoub , Mohammad Yaqub

Prediction tasks in digital pathology are challenging due to the massive size of whole-slide images (WSIs) and the weak nature of training signals. Advances in computing, data availability, and self-supervised learning (SSL) have paved the…

Image and Video Processing · Electrical Eng. & Systems 2026-02-02 Vishwesh Ramanathan , Tony Xu , Pushpak Pati , Faruk Ahmed , Maged Goubran , Anne L. Martel

Accurate survival prediction is essential for personalized cancer treatment. However, genomic data - often a more powerful predictor than pathology data - is costly and inaccessible. We present the cross-modal genomic feature translation…

Image and Video Processing · Electrical Eng. & Systems 2024-11-04 Akhila Krishna , Nikhil Cherian Kurian , Abhijeet Patil , Amruta Parulekar , Amit Sethi

Predictive modeling based on genomic data has gained popularity in biomedical research and clinical practice by allowing researchers and clinicians to identify biomarkers and tailor treatment decisions more efficiently. Analysis…

Machine Learning · Statistics 2021-02-02 Li Zeng , Zhaolong Yu , Yiliang Zhang , Hongyu Zhao

Accurate prediction of cancer progression remains a challenge due to the high heterogeneity of molecular omics data across patients. While biologically informed models have improved the interpretability of these predictions, a persistent…

Machine Learning · Computer Science 2026-04-21 Koushik Howlader , Md Tauhidul Islam , Wei Le

Multimodal survival methods combining gigapixel histology whole-slide images (WSIs) and transcriptomic profiles are particularly promising for patient prognostication and stratification. Current approaches involve tokenizing the WSIs into…

Computer Vision and Pattern Recognition · Computer Science 2024-07-02 Andrew H. Song , Richard J. Chen , Guillaume Jaume , Anurag J. Vaidya , Alexander S. Baras , Faisal Mahmood

The rapidly emerging field of deep learning-based computational pathology has shown promising results in utilizing whole slide images (WSIs) to objectively prognosticate cancer patients. However, most prognostic methods are currently…

Computer Vision and Pattern Recognition · Computer Science 2023-11-21 Mingxin Liu , Yunzan Liu , Hui Cui , Chunquan Li , Jiquan Ma

Multimodal machine learning integrating histopathology and molecular data shows promise for cancer prognostication. We systematically reviewed studies combining whole slide images (WSIs) and high-throughput omics to predict overall…

Quantitative Methods · Quantitative Biology 2025-07-30 Charlotte Jennings , Andrew Broad , Lucy Godson , Emily Clarke , David Westhead , Darren Treanor

Survival analysis plays a vital role in making clinical decisions. However, the models currently in use are often difficult to interpret, which reduces their usefulness in clinical settings. Prototype learning presents a potential solution,…

Computer Vision and Pattern Recognition · Computer Science 2025-10-08 Shuo Jiang , Zhuwen Chen , Liaoman Xu , Yanming Zhu , Changmiao Wang , Jiong Zhang , Feiwei Qin , Yifei Chen , Zhu Zhu

Accurate survival prediction is critical in oncology for prognosis and treatment planning. Traditional approaches often rely on a single data modality, limiting their ability to capture the complexity of tumor biology. To address this…

Machine Learning · Computer Science 2025-07-11 Alba Garrido , Alejandro Almodóvar , Patricia A. Apellániz , Juan Parras , Santiago Zazo

Leveraging multimodal data, particularly the integration of whole-slide histology images (WSIs) and transcriptomic profiles, holds great promise for improving cancer survival prediction. However, excessive redundancy in multimodal data can…

Image and Video Processing · Electrical Eng. & Systems 2025-03-07 Hong Liu , Haosen Yang , Federica Eduati , Josien P. W. Pluim , Mitko Veta

The rapidly emerging field of deep learning-based computational pathology has demonstrated promise in developing objective prognostic models from histology whole slide images. However, most prognostic models are either based on histology or…

Computer Vision and Pattern Recognition · Computer Science 2021-08-06 Richard J. Chen , Ming Y. Lu , Drew F. K. Williamson , Tiffany Y. Chen , Jana Lipkova , Muhammad Shaban , Maha Shady , Mane Williams , Bumjin Joo , Zahra Noor , Faisal Mahmood

Cancer survival prediction requires integrating pathological Whole Slide Images (WSIs) and genomic profiles, a challenging task due to the inherent heterogeneity and the complexity of modeling both inter- and intra-modality interactions.…

Image and Video Processing · Electrical Eng. & Systems 2025-07-08 Mingxin Liu , Chengfei Cai , Jun Li , Pengbo Xu , Jinze Li , Jiquan Ma , Jun Xu

Prognostic task is of great importance as it closely related to the survival analysis of patients, the optimization of treatment plans and the allocation of resources. The existing prognostic models have shown promising results on specific…

Image and Video Processing · Electrical Eng. & Systems 2025-01-14 Binyu Zhang , Shichao Li , Junpeng Jian , Zhu Meng , Limei Guo , Zhicheng Zhao

Multi-modal learning plays a crucial role in cancer diagnosis and prognosis. Current deep learning based multi-modal approaches are often limited by their abilities to model the complex correlations between genomics and histology data,…

Image and Video Processing · Electrical Eng. & Systems 2024-06-21 Yupei Zhang , Xiaofei Wang , Fangliangzi Meng , Jin Tang , Chao Li

Cancer survival prediction using multi-modal medical imaging presents a critical challenge in oncology, mainly due to the vulnerability of deep learning models to noise and protocol variations across imaging centers. Current approaches…

Image and Video Processing · Electrical Eng. & Systems 2025-05-06 Aiman Farooq , Azad Singh , Deepak Mishra , Santanu Chaudhury

Survival analysis, as a challenging task, requires integrating Whole Slide Images (WSIs) and genomic data for comprehensive decision-making. There are two main challenges in this task: significant heterogeneity and complex inter- and…

Image and Video Processing · Electrical Eng. & Systems 2024-06-17 Conghao Xiong , Hao Chen , Hao Zheng , Dong Wei , Yefeng Zheng , Joseph J. Y. Sung , Irwin King

Histo-genomic multimodal survival prediction has garnered growing attention for its remarkable model performance and potential contributions to precision medicine. However, a significant challenge in clinical practice arises when only…

Machine Learning · Computer Science 2025-03-17 Fengchun Liu , Linghan Cai , Zhikang Wang , Zhiyuan Fan , Jin-gang Yu , Hao Chen , Yongbing Zhang

Accurate survival prediction is essential for personalised cancer treatment. We propose ModalSurv, a multimodal deep survival framework integrating clinical, MRI, histopathology, and RNA-sequencing data via modality-specific projections and…

Machine Learning · Computer Science 2025-12-19 Noorul Wahab , Ethar Alzaid , Jiaqi Lv , Fayyaz Minhas , Adam Shephard , Shan E Ahmed Raza