English
Related papers

Related papers: No Modality Left Behind: Adapting to Missing Modal…

200 papers

Survival prediction is a major concern for cancer management. Deep survival models based on deep learning have been widely adopted to perform end-to-end survival prediction from medical images. Recent deep survival models achieved promising…

Image and Video Processing · Electrical Eng. & Systems 2024-10-17 Mingyuan Meng , Bingxin Gu , Michael Fulham , Shaoli Song , Dagan Feng , Lei Bi , Jinman Kim

Missing modalities pose a major issue in Alzheimer's Disease (AD) diagnosis, as many subjects lack full imaging data due to cost and clinical constraints. While multi-modal learning leverages complementary information, most existing methods…

Image and Video Processing · Electrical Eng. & Systems 2025-08-28 Yanfei Li , Teng Yin , Wenyi Shang , Jingyu Liu , Xi Wang , Kaiyang Zhao

Multimodal magnetic resonance imaging (MRI) provides complementary information for sub-region analysis of brain tumors. Plenty of methods have been proposed for automatic brain tumor segmentation using four common MRI modalities and…

Image and Video Processing · Electrical Eng. & Systems 2023-03-10 Hong Liu , Dong Wei , Donghuan Lu , Jinghan Sun , Liansheng Wang , Yefeng Zheng

Accurate brain tumor segmentation using multiparametric MRI is critical for effective treatment planning. However, in clinical settings, complete acquisition of all MRI sequences is not always possible. The absence of certain MRI modalities…

Computer Vision and Pattern Recognition · Computer Science 2026-05-22 Danish Ali , Ajmal Mian , Naveed Akhtar , Ghulam Mubashar Hassan

Multi-modal brain tumor segmentation is critical for clinical diagnosis, and it requires accurate identification of distinct internal anatomical subregions. While the recent prompt-based segmentation paradigms enable interactive experiences…

Computer Vision and Pattern Recognition · Computer Science 2025-10-14 Yuxiang Luo , Qing Xu , Hai Huang , Yuqi Ouyang , Zhen Chen , Wenting Duan

The successful adaptation of foundation models to multi-modal medical imaging is a critical yet unresolved challenge. Existing models often struggle to effectively fuse information from multiple sources and adapt to the heterogeneous nature…

Computer Vision and Pattern Recognition · Computer Science 2026-01-23 Shadi Alijani , Fereshteh Aghaee Meibodi , Homayoun Najjaran

The joint use of multiple imaging modalities for medical image segmentation has been widely studied in recent years. The fusion of information from different modalities has demonstrated to improve the segmentation accuracy, with respect to…

Computer Vision and Pattern Recognition · Computer Science 2021-06-18 Minhao Hu , Matthis Maillard , Ya Zhang , Tommaso Ciceri , Giammarco La Barbera , Isabelle Bloch , Pietro Gori

The joint utilization of diverse data sources for medical imaging segmentation has emerged as a crucial area of research, aiming to address challenges such as data heterogeneity, domain shift, and data quality discrepancies. Integrating…

Computer Vision and Pattern Recognition · Computer Science 2024-04-04 Eddardaa B. Loussaief , Mohammed Ayad , Domenc Puig , Hatem A. Rashwan

Radiologists must utilize multiple modal images for tumor segmentation and diagnosis due to the limitations of medical imaging and the diversity of tumor signals. This leads to the development of multimodal learning in segmentation.…

Computer Vision and Pattern Recognition · Computer Science 2024-07-11 Chuyun Shen , Wenhao Li , Haoqing Chen , Xiaoling Wang , Fengping Zhu , Yuxin Li , Xiangfeng Wang , Bo Jin

Accurate brain tumor segmentation from Magnetic Resonance Imaging (MRI) is desirable to joint learning of multimodal images. However, in clinical practice, it is not always possible to acquire a complete set of MRIs, and the problem of…

Image and Video Processing · Electrical Eng. & Systems 2022-08-05 Yao Zhang , Nanjun He , Jiawei Yang , Yuexiang Li , Dong Wei , Yawen Huang , Yang Zhang , Zhiqiang He , Yefeng Zheng

Background: Brain tumor segmentation has a significant impact on the diagnosis and treatment of brain tumors. Accurate brain tumor segmentation remains challenging due to their irregular shapes, vague boundaries, and high variability.…

Computer Vision and Pattern Recognition · Computer Science 2025-05-07 Zhanyuan Jia , Ni Yao , Danyang Sun , Chuang Han , Yanting Li , Jiaofen Nan , Fubao Zhu , Chen Zhao , Weihua Zhou

Multi-modal brain images from MRI scans are widely used in clinical diagnosis to provide complementary information from different modalities. However, obtaining fully paired multi-modal images in practice is challenging due to various…

Image and Video Processing · Electrical Eng. & Systems 2024-04-25 Chuan Huang , Jia Wei , Rui Li

The segmentation of brain tumors in multimodal MRIs is one of the most challenging tasks in medical image analysis. The recent state of the art algorithms solving this task is based on machine learning approaches and deep learning in…

Image and Video Processing · Electrical Eng. & Systems 2020-02-11 Dmitrii Lachinov , Elena Shipunova , Vadim Turlapov

Multimodal learning has shown great potentials in numerous scenes and attracts increasing interest recently. However, it often encounters the problem of missing modality data and thus suffers severe performance degradation in practice. To…

Computer Vision and Pattern Recognition · Computer Science 2023-04-18 Shicai Wei , Yang Luo , Chunbo Luo

Multi-modal MR imaging is routinely used in clinical practice to diagnose and investigate brain tumors by providing rich complementary information. Previous multi-modal MRI segmentation methods usually perform modal fusion by concatenating…

Image and Video Processing · Electrical Eng. & Systems 2022-09-01 Zhaohu Xing , Lequan Yu , Liang Wan , Tong Han , Lei Zhu

Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. Accurate identification of the type and grade of tumor in the early stages plays an important role in choosing a precise…

Computer Vision and Pattern Recognition · Computer Science 2024-01-18 Dunyuan Xu , Xi Wang , Jinyue Cai , Pheng-Ann Heng

Multimodal fusion leverages information across modalities to learn better feature representations with the goal of improving performance in fusion-based tasks. However, multimodal datasets, especially in medical settings, are typically…

Machine Learning · Computer Science 2025-02-05 Alejandro Guerra-Manzanares , Farah E. Shamout

Image modality is not perfect as it often fails in certain conditions, e.g., night and fast motion. This significantly limits the robustness and versatility of existing multi-modal (i.e., Image+X) semantic segmentation methods when…

Computer Vision and Pattern Recognition · Computer Science 2024-07-17 Xu Zheng , Yuanhuiyi Lyu , Lin Wang

Multi-modal magnetic resonance imaging (MRI) is a crucial method for analyzing the human brain. It is usually used for diagnosing diseases and for making valuable decisions regarding the treatments - for instance, checking for gliomas in…

Image and Video Processing · Electrical Eng. & Systems 2021-05-26 Ashwin Nalwade , Jackie Kisa

Traditional brain lesion segmentation models for multi-modal MRI are typically tailored to specific pathologies, relying on datasets with predefined modalities. Adapting to new MRI modalities or pathologies often requires training separate…

Image and Video Processing · Electrical Eng. & Systems 2025-07-25 Yousef Sadegheih , Pratibha Kumari , Dorit Merhof