English
Related papers

Related papers: UNICORN: A Deep Learning Model for Integrating Mul…

200 papers

In this paper, an innovative multi-modal deep learning model is proposed to deeply integrate heterogeneous information from medical images and clinical reports. First, for medical images, convolutional neural networks were used to extract…

Machine Learning · Computer Science 2024-05-29 Ziyan Yao , Fei Lin , Sheng Chai , Weijie He , Lu Dai , Xinghui Fei

Staining is essential in cell imaging and medical diagnostics but poses significant challenges, including high cost, time consumption, labor intensity, and irreversible tissue alterations. Recent advances in deep learning have enabled…

Computer Vision and Pattern Recognition · Computer Science 2025-04-15 Ziwang Xu , Lanqing Guo , Satoshi Tsutsui , Shuyan Zhang , Alex C. Kot , Bihan Wen

Due to its complexity, graph learning-based multi-modal integration and classification is one of the most challenging obstacles for disease prediction. To effectively offset the negative impact between modalities in the process of…

Machine Learning · Computer Science 2025-02-14 Jin Liu , Junbin Mao , Hanhe Lin , Hulin Kuang , Shirui Pan , Xusheng Wu , Shan Xie , Fei Liu , Yi Pan

To accurately analyze changes of anatomical structures in longitudinal imaging studies, consistent segmentation across multiple time-points is required. Existing solutions often involve independent registration and segmentation components.…

Image and Video Processing · Electrical Eng. & Systems 2019-08-28 Bo Li , Wiro Niessen , Stefan Klein , Marius de Groot , Arfan Ikram , Meike Vernooij , Esther Bron

Machine learning methods in healthcare have traditionally focused on using data from a single modality, limiting their ability to effectively replicate the clinical practice of integrating multiple sources of information for improved…

Machine Learning · Computer Science 2024-02-13 Felix Krones , Umar Marikkar , Guy Parsons , Adam Szmul , Adam Mahdi

Multimodal medical imaging plays a pivotal role in clinical diagnosis and research, as it combines information from various imaging modalities to provide a more comprehensive understanding of the underlying pathology. Recently, deep…

Computer Vision and Pattern Recognition · Computer Science 2024-04-24 Yihao Li , Mostafa El Habib Daho , Pierre-Henri Conze , Rachid Zeghlache , Hugo Le Boité , Ramin Tadayoni , Béatrice Cochener , Mathieu Lamard , Gwenolé Quellec

Existing multi-modal approaches primarily focus on enhancing multi-label skin lesion classification performance through advanced fusion modules, often neglecting the associated rise in parameters. In clinical settings, both clinical and…

Image and Video Processing · Electrical Eng. & Systems 2024-07-16 Peng Tang , Tobias Lasser

The accuracy of predictive models for solitary pulmonary nodule (SPN) diagnosis can be greatly increased by incorporating repeat imaging and medical context, such as electronic health records (EHRs). However, clinically routine modalities…

Artificial intelligence holds strong potential to support clinical decision making in intensive care units where timely and accurate risk assessment is critical. However, many existing models focus on isolated outcomes or limited data…

The rapid development of diagnostic technologies in healthcare is leading to higher requirements for physicians to handle and integrate the heterogeneous, yet complementary data that are produced during routine practice. For instance, the…

Machine Learning · Computer Science 2023-01-30 Can Cui , Haichun Yang , Yaohong Wang , Shilin Zhao , Zuhayr Asad , Lori A. Coburn , Keith T. Wilson , Bennett A. Landman , Yuankai Huo

Integrating multi-omics data, such as DNA methylation, mRNA expression, and microRNA (miRNA) expression, offers a comprehensive view of the biological mechanisms underlying disease. However, the high dimensionality of multi-omics data, the…

Machine Learning · Computer Science 2026-02-12 Tiantian Yang , Zhiqian Chen

In this study, we introduce a multi-modal approach that efficiently integrates multi-scale clinical and dermoscopy features within a single network, thereby substantially reducing model parameters. The proposed method includes three novel…

Image and Video Processing · Electrical Eng. & Systems 2024-03-31 Peng Tang , Tobias Lasser

Acute and chronic wounds are a challenge to healthcare systems around the world and affect many people's lives annually. Wound classification is a key step in wound diagnosis that would help clinicians to identify an optimal treatment…

Computer Vision and Pattern Recognition · Computer Science 2021-06-07 Behrouz Rostami , D. M. Anisuzzaman , Chuanbo Wang , Sandeep Gopalakrishnan , Jeffrey Niezgoda , Zeyun Yu

Multi-modal skin lesion diagnosis (MSLD) has achieved remarkable success by modern computer-aided diagnosis (CAD) technology based on deep convolutions. However, the information aggregation across modalities in MSLD remains challenging due…

Computer Vision and Pattern Recognition · Computer Science 2023-03-03 Yilan Zhang , Fengying Xie , Jianqi Chen

Reliable induction motor (IM) fault diagnosis is vital for industrial safety and operational continuity, mitigating costly unplanned downtime. Conventional approaches often struggle to capture complex multimodal signal relationships, are…

Artificial Intelligence · Computer Science 2025-10-20 Usman Ali , Ali Zia , Waqas Ali , Umer Ramzan , Abdul Rehman , Muhammad Tayyab Chaudhry , Wei Xiang

Multi-modal learning plays a crucial role in cancer diagnosis and prognosis. Current deep learning based multi-modal approaches are often limited by their abilities to model the complex correlations between genomics and histology data,…

Image and Video Processing · Electrical Eng. & Systems 2024-06-21 Yupei Zhang , Xiaofei Wang , Fangliangzi Meng , Jin Tang , Chao Li

Characterising the heterogeneous presentation of Parkinson's disease (PD) requires integrating biological and clinical markers within a unified predictive framework. While multimodal data provide complementary information, many existing…

Machine Learning · Computer Science 2026-01-05 Dristi Datta , Tanmoy Debnath , Minh Chau , Manoranjan Paul , Gourab Adhikary , Md Geaur Rahman

Progress in a research field can be hard to assess, in particular when many concurrent methods are proposed in a short period of time. This is the case in digital pathology, where many foundation models have been released recently to serve…

Computer Vision and Pattern Recognition · Computer Science 2026-02-18 Pierre Marza , Leo Fillioux , Sofiène Boutaj , Kunal Mahatha , Christian Desrosiers , Pablo Piantanida , Jose Dolz , Stergios Christodoulidis , Maria Vakalopoulou

This paper proposes a high-precision semantic segmentation method based on an improved TransUNet architecture to address the challenges of complex lesion structures, blurred boundaries, and significant scale variations in skin lesion…

Image and Video Processing · Electrical Eng. & Systems 2025-08-21 Xin Wang , Xiaopei Zhang , Xingang Wang

Stroke is a major public health problem, affecting millions worldwide. Deep learning has recently demonstrated promise for enhancing the diagnosis and risk prediction of stroke. However, existing methods rely on costly medical imaging…

Image and Video Processing · Electrical Eng. & Systems 2025-12-17 Saeed Shurrab , Aadim Nepal , Terrence J. Lee-St. John , Nicola G. Ghazi , Bartlomiej Piechowski-Jozwiak , Farah E. Shamout