English
Related papers

Related papers: Multi-modal Learning with Missing Modality in Pred…

200 papers

Multimodal fusion leverages information across modalities to learn better feature representations with the goal of improving performance in fusion-based tasks. However, multimodal datasets, especially in medical settings, are typically…

Machine Learning · Computer Science 2025-02-05 Alejandro Guerra-Manzanares , Farah E. Shamout

Breast cancer is the most prevalent cancer in women worldwide. Histopathology image analysis serves as the gold standard for cancer diagnosis. In this regard, whole-slide imaging (WSI), a revolutionary technology in digital pathology,…

Methodology · Statistics 2026-04-21 Baichen Yu , Xuetong Li , Jing Zhou , Hansheng Wang

Brain tumor segmentation remains a significant challenge, particularly in the context of multi-modal magnetic resonance imaging (MRI) where missing modality images are common in clinical settings, leading to reduced segmentation accuracy.…

Image and Video Processing · Electrical Eng. & Systems 2024-06-14 Zhongao Sun , Jiameng Li , Yuhan Wang , Jiarong Cheng , Qing Zhou , Chun Li

In this work, we address the problem of learning an ensemble of specialist networks using multimodal data, while considering the realistic and challenging scenario of possible missing modalities at test time. Our goal is to leverage the…

Computer Vision and Pattern Recognition · Computer Science 2019-12-24 Nuno C. Garcia , Sarah Adel Bargal , Vitaly Ablavsky , Pietro Morerio , Vittorio Murino , Stan Sclaroff

Although the combined treatment of surgery, radiotherapy, chemotherapy, and emerging target therapy has significantly improved the outcomes of patients with head and neck cancer, distant metastasis remains the leading cause of treatment…

Quantitative Methods · Quantitative Biology 2025-12-02 Nuo Tong , Changhao Liu , Zizhao Tang , Feifan Sun , Yingping Li , Shuiping Gou , Mei Shi

In histopathology, intelligent diagnosis of Whole Slide Images (WSIs) is essential for automating and objectifying diagnoses, reducing the workload of pathologists. However, diagnostic models often face the challenge of forgetting…

Computer Vision and Pattern Recognition · Computer Science 2025-03-26 Weixi Zheng , Aoling Huang , Jingping Yuan , Haoyu Zhao , Zhou Zhao , Yongchao Xu , Thierry Géraud

Multimodal deep learning (MDL) has emerged as a transformative approach in computational pathology. By integrating complementary information from multiple data sources, MDL models have demonstrated superior predictive performance across…

Quantitative Methods · Quantitative Biology 2025-11-17 Seth Alain Chang , Muhammad Mueez Amjad , Noorul Wahab , Ethar Alzaid , Nasir Rajpoot , Adam Shephard

Automatic lymph node segmentation is the cornerstone for advances in computer vision tasks for early detection and staging of cancer. Traditional segmentation methods are constrained by manual delineation and variability in operator…

Image and Video Processing · Electrical Eng. & Systems 2025-06-11 Jingguo Qu , Xinyang Han , Man-Lik Chui , Yao Pu , Simon Takadiyi Gunda , Ziman Chen , Jing Qin , Ann Dorothy King , Winnie Chiu-Wing Chu , Jing Cai , Michael Tin-Cheung Ying

In computational pathology, multiple instance learning (MIL) is widely used to circumvent the computational impasse in giga-pixel whole slide image (WSI) analysis. It usually consists of two stages: patch-level feature extraction and…

Image and Video Processing · Electrical Eng. & Systems 2023-09-21 Beidi Zhao , Wenlong Deng , Zi Han , Li , Chen Zhou , Zuhua Gao , Gang Wang , Xiaoxiao Li

Medical imaging is a cornerstone of therapy and diagnosis in modern medicine. However, the choice of imaging modality for a particular theranostic task typically involves trade-offs between the feasibility of using a particular modality…

Computer Vision and Pattern Recognition · Computer Science 2022-07-22 Mayur Mallya , Ghassan Hamarneh

Acquiring properly annotated data is expensive in the medical field as it requires experts, time-consuming protocols, and rigorous validation. Active learning attempts to minimize the need for large annotated samples by actively sampling…

Image and Video Processing · Electrical Eng. & Systems 2023-06-22 Bidur Khanal , Binod Bhattarai , Bishesh Khanal , Danail Stoyanov , Cristian A. Linte

Self-supervised learning is an efficient pre-training method for medical image analysis. However, current research is mostly confined to specific-modality data pre-training, consuming considerable time and resources without achieving…

Computer Vision and Pattern Recognition · Computer Science 2023-12-01 Yiwen Ye , Yutong Xie , Jianpeng Zhang , Ziyang Chen , Qi Wu , Yong Xia

It is imperative that breast cancer is detected precisely and timely to improve patient outcomes. Diagnostic methodologies have traditionally relied on unimodal approaches; however, medical data analytics is integrating diverse data sources…

Machine Learning · Computer Science 2024-06-21 Faseela Abdullakutty , Younes Akbari , Somaya Al-Maadeed , Ahmed Bouridane , Rifat Hamoudi

Multimodal Machine Learning offers a holistic view of a patient's status, integrating structured and unstructured data from electronic health records (EHR). We propose a framework to predict metastasis risk one month prior to diagnosis,…

Machine Learning · Computer Science 2026-04-01 Franco Rugolon , Korbinian Randl , Braslav Jovanovic , Ioanna Miliou , Panagiotis Papapetrou

Multimodal learning leverages complementary information derived from different modalities, thereby enhancing performance in medical image segmentation. However, prevailing multimodal learning methods heavily rely on extensive well-annotated…

Computer Vision and Pattern Recognition · Computer Science 2024-09-05 Xiaogen Zhou , Yiyou Sun , Min Deng , Winnie Chiu Wing Chu , Qi Dou

Multimodal magnetic resonance imaging (MRI) is crucial for brain tumor segmentation, with many methods leveraging its four key modalities to capture complementary information for effective sub-region analysis. However, the absence of…

Artificial Intelligence · Computer Science 2026-05-19 Sha Tao , Jiao Pan , Yu Guo , Chao Yao

Medical multimodal learning faces significant challenges with missing modalities prevalent in clinical practice. Existing approaches assume equal contribution of modality and random missing patterns, neglecting inherent uncertainty in…

Machine Learning · Computer Science 2026-01-30 Linxiao Gong , Yang Liu , Lianlong Sun , Yulai Bi , Jing Liu , Xiaoguang Zhu

A key challenge in learning from multimodal biological data is missing modalities, where data from one or more modalities are absent for some patients. Existing approaches either exclude patients with missing modalities, impute missing…

Machine Learning · Computer Science 2026-05-19 Sina Tabakhi , Chen , Chen , Haiping Lu

Melanoma detection is vital for early diagnosis and effective treatment. While deep learning models on dermoscopic images have shown promise, they require specialized equipment, limiting their use in broader clinical settings. This study…

Computer Vision and Pattern Recognition · Computer Science 2026-05-20 Volodymyr Sydorskyi , Igor Krashenyi , Oleksii Yakubenko

Automatic lymph node (LN) segmentation and detection for cancer staging are critical. In clinical practice, computed tomography (CT) and positron emission tomography (PET) imaging detect abnormal LNs. Despite its low contrast and variety in…

Image and Video Processing · Electrical Eng. & Systems 2022-12-23 Al-Akhir Nayan , Boonserm Kijsirikul , Yuji Iwahori