English
Related papers

Related papers: Multimodal Learning for Scalable Representation of…

200 papers

Multimodal learning, integrating histology images and genomics, promises to enhance precision oncology with comprehensive views at microscopic and molecular levels. However, existing methods may not sufficiently model the shared or…

Computer Vision and Pattern Recognition · Computer Science 2024-06-12 Huahui Yi , Xiaofei Wang , Kang Li , Chao Li

Recent advances in artificial intelligence, particularly large language models LLMs, have shown promising capabilities in transforming rare disease research. This survey paper explores the integration of LLMs in the analysis of rare…

Computation and Language · Computer Science 2025-05-26 Valentina Carbonari , Pierangelo Veltri , Pietro Hiram Guzzi

The early detection and nuanced subtype classification of non-small cell lung cancer (NSCLC), a predominant cause of cancer mortality worldwide, is a critical and complex issue. In this paper, we introduce an innovative integration of…

Image and Video Processing · Electrical Eng. & Systems 2024-10-08 Salma Hassan , Hamad Al Hammadi , Ibrahim Mohammed , Muhammad Haris Khan

The rise of In-Context Learning (ICL) for universal medical image segmentation has introduced an unprecedented demand for large-scale, diverse datasets for training, exacerbating the long-standing problem of data scarcity. While data…

Computer Vision and Pattern Recognition · Computer Science 2025-09-25 Jiesi Hu , Yanwu Yang , Zhiyu Ye , Chenfei Ye , Hanyang Peng , Jianfeng Cao , Ting Ma

Current artificial intelligence models for medical imaging are predominantly single modality and single disease. Attempts to create multimodal and multi-disease models have resulted in inconsistent clinical accuracy. Furthermore, training…

Recent advancements in the acquisition of various brain data sources have created new opportunities for integrating multimodal brain data to assist in early detection of complex brain disorders. However, current data integration approaches…

Image and Video Processing · Electrical Eng. & Systems 2023-05-26 Reza Shirkavand , Liang Zhan , Heng Huang , Li Shen , Paul M. Thompson

Medical imaging is a cornerstone of therapy and diagnosis in modern medicine. However, the choice of imaging modality for a particular theranostic task typically involves trade-offs between the feasibility of using a particular modality…

Computer Vision and Pattern Recognition · Computer Science 2022-07-22 Mayur Mallya , Ghassan Hamarneh

Medical patient data is always multimodal. Images, text, age, gender, histopathological data are only few examples for different modalities in this context. Processing and integrating this multimodal data with deep learning based methods is…

Artificial Intelligence · Computer Science 2025-09-11 Christian Gapp , Elias Tappeiner , Martin Welk , Rainer Schubert

Precision medicine fundamentally aims to establish causality between dysregulated biochemical mechanisms and cancer subtypes. Omics-based cancer subtyping has emerged as a revolutionary approach, as different level of omics records the…

Genomics · Quantitative Biology 2023-08-25 Ziwei Yang , Zheng Chen , Yasuko Matsubara , Yasushi Sakurai

Self-supervised learning has recently gained growing interest in molecular modeling for scientific tasks such as AI-assisted drug discovery. Current studies consider leveraging both 2D and 3D molecular structures for representation…

Machine Learning · Computer Science 2023-10-10 Qiying Yu , Yudi Zhang , Yuyan Ni , Shikun Feng , Yanyan Lan , Hao Zhou , Jingjing Liu

Routine clinical visits of a patient produce not only image data, but also non-image data containing clinical information regarding the patient, i.e., medical data is multi-modal in nature. Such heterogeneous modalities offer different and…

Computer Vision and Pattern Recognition · Computer Science 2023-03-16 Sein Kim , Namkyeong Lee , Junseok Lee , Dongmin Hyun , Chanyoung Park

The increasing interest in developing Artificial Intelligence applications in the medical domain, suffers from the lack of high-quality data set, mainly due to privacy-related issues. In addition, the recent increase in Vision Language…

Databases · Computer Science 2025-07-18 Irene Siragusa , Salvatore Contino , Massimo La Ciura , Rosario Alicata , Roberto Pirrone

Machine learning holds promise for advancing clinical decision support, yet it remains unclear when multimodal learning truly helps in practice, particularly under modality missingness and fairness constraints. In this work, we conduct a…

Machine Learning · Computer Science 2026-03-02 Kejing Yin , Haizhou Xu , Wenfang Yao , Chen Liu , Zijie Chen , Yui Haang Cheung , William K. Cheung , Jing Qin

With the ongoing development of deep learning, an increasing number of AI models have surpassed the performance levels of human clinical practitioners. However, the prevalence of AI diagnostic products in actual clinical practice remains…

Computer Vision and Pattern Recognition · Computer Science 2024-03-11 Chenglong Wang , Yinqiao Yi , Yida Wang , Chengxiu Zhang , Yun Liu , Kensaku Mori , Mei Yuan , Guang Yang

Rapid technological advances have allowed for molecular profiling across multiple omics domains from a single sample for clinical decision making in many diseases, especially cancer. As tumor development and progression are dynamic…

Methodology · Statistics 2022-02-11 Dongyan Yan , Subharup Guha

Multimodal machine learning integrating histopathology and molecular data shows promise for cancer prognostication. We systematically reviewed studies combining whole slide images (WSIs) and high-throughput omics to predict overall…

Quantitative Methods · Quantitative Biology 2025-07-30 Charlotte Jennings , Andrew Broad , Lucy Godson , Emily Clarke , David Westhead , Darren Treanor

The burgeoning discipline of computational pathology shows promise in harnessing whole slide images (WSIs) to quantify morphological heterogeneity and develop objective prognostic modes for human cancers. However, progress is impeded by the…

Computer Vision and Pattern Recognition · Computer Science 2025-10-20 Chao Tu , Kun Huang , Jie Zhang , Qianjin Feng , Yu Zhang , Zhenyuan Ning

Digital pathology has fundamentally altered diagnostic workflows by enabling the computational analysis of gigapixel Whole Slide Images (WSIs), yet effectively deciphering their complex tumor microenvironments remains a formidable…

Computer Vision and Pattern Recognition · Computer Science 2026-04-28 Xuemei Qiu , Dawei Fan , Yebin Huang , Yanping Chen , Lifang Wei

Multi-disease diagnosis using multi-modal data like electronic health records and medical imaging is a critical clinical task. Although existing deep learning methods have achieved initial success in this area, a significant gap persists…

Multimedia · Computer Science 2025-09-22 Yueheng Jiang , Peng Zhang

While state-of-the-art models for breast cancer detection leverage multi-view mammograms for enhanced diagnostic accuracy, they often focus solely on visual mammography data. However, radiologists document valuable lesion descriptors that…

Computer Vision and Pattern Recognition · Computer Science 2024-11-19 Gil Ben-Artzi , Feras Daragma , Shahar Mahpod