English
Related papers

Related papers: A Modality-agnostic Multi-task Foundation Model fo…

200 papers

Precise prognostic modeling of glioblastoma (GBM) under varying treatment interventions is essential for optimizing clinical outcomes. While generative AI has shown promise in simulating GBM evolution, existing methods typically treat…

Computer Vision and Pattern Recognition · Computer Science 2026-03-10 Chenhui Wang , Boyun Zheng , Liuxin Bao , Zhihao Peng , Peter Y. M. Woo , Hongming Shan , Yixuan Yuan

Magnetic resonance imaging~(MRI) have played a crucial role in brain disease diagnosis, with which a range of computer-aided artificial intelligence methods have been proposed. However, the early explorations usually focus on the limited…

Computer Vision and Pattern Recognition · Computer Science 2023-09-14 Jiayu Lei , Lisong Dai , Haoyun Jiang , Chaoyi Wu , Xiaoman Zhang , Yao Zhang , Jiangchao Yao , Weidi Xie , Yanyong Zhang , Yuehua Li , Ya Zhang , Yanfeng Wang

Brain imaging analysis is crucial for diagnosing and treating brain disorders, and multimodal large language models (MLLMs) are increasingly supporting it. However, current brain imaging visual question-answering (VQA) benchmarks either…

Computer Vision and Pattern Recognition · Computer Science 2025-12-29 Zhihao Peng , Cheng Wang , Shengyuan Liu , Zhiying Liang , Zanting Ye , Minjie Ju , PeterYM Woo , Yixuan Yuan

Vision Foundation Models (VFMs) have become the cornerstone of modern computer vision, offering robust representations across a wide array of tasks. While recent advances allow these models to handle varying input sizes during training,…

Computer Vision and Pattern Recognition · Computer Science 2026-04-06 Bocheng Zou , Mu Cai , Mark Stanley , Dingfu Lu , Yong Jae Lee

Vision foundation models like DINOv2 demonstrate remarkable potential in medical imaging despite their origin in natural image domains. However, their design inherently works best for uni-modal image analysis, limiting their effectiveness…

Image and Video Processing · Electrical Eng. & Systems 2025-09-09 Daniel Scholz , Ayhan Can Erdur , Viktoria Ehm , Anke Meyer-Baese , Jan C. Peeken , Daniel Rueckert , Benedikt Wiestler

The field of computer vision is undergoing a paradigm shift toward large-scale foundation model pre-training via self-supervised learning (SSL). Leveraging large volumes of unlabeled brain MRI data, such models can learn anatomical priors…

Image and Video Processing · Electrical Eng. & Systems 2026-01-15 Petros Koutsouvelis , Matej Gazda , Leroy Volmer , Sina Amirrajab , Kamil Barbierik , Branislav Setlak , Jakub Gazda , Peter Drotar

Deformable image registration plays a fundamental role in medical image analysis by enabling spatial alignment of anatomical structures across subjects. While recent deep learning-based approaches have significantly improved computational…

Image and Video Processing · Electrical Eng. & Systems 2026-03-24 Jiaqi Shang , Haojin Wu , Yinyi Lai , Zongyu Li , Chenghao Zhang , Jia Guo

Multimodal MRI is essential for brain tumor segmentation, yet missing modalities in clinical practice cause existing methods to exhibit >40% performance variance across modality combinations, rendering them clinically unreliable. We propose…

Image and Video Processing · Electrical Eng. & Systems 2026-01-28 Chengxiang Guo , Jian Wang , Junhua Fei , Xiao Li , Chunling Chen , Yun Jin

Brain MRI underpins a wide range of neuroscientific and clinical applications, yet most learning-based methods remain task-specific and require substantial labeled data. Here we show that a single self-supervised representation can…

Machine Learning · Computer Science 2026-05-27 Yizhou Wu , Shansong Wang , Yuheng Li , Mojtaba Safari , Mingzhe Hu , Chih-Wei Chang , Harini Veeraraghavan , Xiaofeng Yang

Multi-modal MRIs are widely used in neuroimaging applications since different MR sequences provide complementary information about brain structures. Recent works have suggested that multi-modal deep learning analysis can benefit from…

Computer Vision and Pattern Recognition · Computer Science 2021-06-14 Jiahong Ouyang , Ehsan Adeli , Kilian M. Pohl , Qingyu Zhao , Greg Zaharchuk

Brain tumor segmentation is often based on multiple magnetic resonance imaging (MRI). However, in clinical practice, certain modalities of MRI may be missing, which presents an even more difficult scenario. To cope with this challenge,…

Image and Video Processing · Electrical Eng. & Systems 2025-01-16 Tianyi Liu , Zhaorui Tan , Haochuan Jiang , Xi Yang , Kaizhu Huang

Radio-Frequency (RF)-based Human Activity Recognition (HAR) rises as a promising solution for applications unamenable to techniques requiring computer visions. However, the scarcity of labeled RF data due to their non-interpretable nature…

Computer Vision and Pattern Recognition · Computer Science 2024-10-29 Yuxuan Weng , Guoquan Wu , Tianyue Zheng , Yanbing Yang , Jun Luo

We present brat (brain report alignment transformer), a multi-view representation learning framework for brain magnetic resonance imaging (MRI) trained on MRIs paired with clinical reports. Brain MRIs present unique challenges due to the…

Computer Vision and Pattern Recognition · Computer Science 2025-12-23 Maxime Kayser , Maksim Gridnev , Wanting Wang , Max Bain , Aneesh Rangnekar , Avijit Chatterjee , Aleksandr Petrov , Harini Veeraraghavan , Nathaniel C. Swinburne

In clinical practice, full imaging is not always feasible, often due to complex acquisition protocols, stringent privacy regulations, or specific clinical needs. However, missing MR modalities pose significant challenges for tasks like…

Computer Vision and Pattern Recognition · Computer Science 2025-01-23 Aghiles Kebaili , Jérôme Lapuyade-Lahorgue , Pierre Vera , Su Ruan

Multimodal medical image fusion helps in combining contrasting features from two or more input imaging modalities to represent fused information in a single image. One of the pivotal clinical applications of medical image fusion is the…

Image and Video Processing · Electrical Eng. & Systems 2019-09-20 Nishant Kumar , Nico Hoffmann , Martin Oelschlägel , Edmund Koch , Matthias Kirsch , Stefan Gumhold

We present a cross-modality generation framework that learns to generate translated modalities from given modalities in MR images without real acquisition. Our proposed method performs NeuroImage-to-NeuroImage translation (abbreviated as…

Computer Vision and Pattern Recognition · Computer Science 2018-09-12 Qianye Yang , Nannan Li , Zixu Zhao , Xingyu Fan , Eric I-Chao Chang , Yan Xu

Developing Foundation Models for medical image analysis is essential to overcome the unique challenges of radiological tasks. The first challenges of this kind for 3D brain MRI, SSL3D and FOMO25, were held at MICCAI 2025. Our solution…

Computer Vision and Pattern Recognition · Computer Science 2026-01-21 Pedro M. Gordaliza , Jaume Banus , Benoît Gérin , Maxence Wynen , Nataliia Molchanova , Jonas Richiardi , Meritxell Bach Cuadra

While Multimodal Large Language Models (MLLMs) excel at single-image understanding, they exhibit significantly degraded performance in multi-image reasoning scenarios. Multi-image reasoning presents fundamental challenges including complex…

Computer Vision and Pattern Recognition · Computer Science 2026-01-13 Jianghao Yin , Qingbin Li , Kun Sun , Cheng Ding , Jie Wang , Qin Chen , Jie Zhou , Nan Wang , Changqing Li , Pei Wu , Jian Xu , Zheming Yang , Liang He

We present Brain Harmony (BrainHarmonix), the first multimodal brain foundation model that unifies structural morphology and functional dynamics into compact 1D token representations. The model was pretrained on two of the largest…

Multi-contrast magnetic resonance (MR) image registration is useful in the clinic to achieve fast and accurate imaging-based disease diagnosis and treatment planning. Nevertheless, the efficiency and performance of the existing registration…

Image and Video Processing · Electrical Eng. & Systems 2021-02-17 Weijian Huang , Hao Yang , Xinfeng Liu , Cheng Li , Ian Zhang , Rongpin Wang , Hairong Zheng , Shanshan Wang