English
Related papers

Related papers: Transformers for Multimodal Brain State Decoding: …

200 papers

Neural decoding, the process of understanding how brain activity corresponds to different stimuli, has been a primary objective in cognitive sciences. Over the past three decades, advances in functional Magnetic Resonance Imaging (fMRI) and…

Computer Vision and Pattern Recognition · Computer Science 2026-01-28 Yanchen Wang , Adam Turnbull , Tiange Xiang , Yunlong Xu , Sa Zhou , Adnan Masoud , Shekoofeh Azizi , Feng Vankee Lin , Ehsan Adeli

Modern deep learning algorithms geared towards clinical adaption rely on a significant amount of high fidelity labeled data. Low-resource settings pose challenges like acquiring high fidelity data and becomes the bottleneck for developing…

Image and Video Processing · Electrical Eng. & Systems 2020-03-23 Szu-Yeu Hu , Shuhang Wang , Wei-Hung Weng , JingChao Wang , XiaoHong Wang , Arinc Ozturk , Qian Li , Viksit Kumar , Anthony E. Samir

We present a sequential transfer learning framework for transformers on functional Magnetic Resonance Imaging (fMRI) data and demonstrate its significant benefits for decoding musical timbre. In the first of two phases, we pre-train our…

Quantitative Methods · Quantitative Biology 2023-05-23 Sean Paulsen , Michael Casey

Compressed sensing MRI seeks to accelerate MRI acquisition processes by sampling fewer k-space measurements and then reconstructing the missing data algorithmically. The success of these approaches often relies on strong priors or learned…

Computer Vision and Pattern Recognition · Computer Science 2025-01-10 Hyungjin Chung , Dohun Lee , Zihui Wu , Byung-Hoon Kim , Katherine L. Bouman , Jong Chul Ye

Diffusion MRI is a powerful tool that serves as a bridge between brain microstructure and cognition. Recent advancements in cognitive neuroscience have highlighted the persistent challenge of understanding how individual differences in…

Neurons and Cognition · Quantitative Biology 2024-09-25 Gabriela Gómez Jiménez , Demian Wassermann

Deformable image registration is fundamental to longitudinal and population analysis. Geometric alignment of the infant brain MR images is challenging, owing to rapid changes in image appearance in association with brain development. In…

Computer Vision and Pattern Recognition · Computer Science 2020-07-07 Dongming Wei , Sahar Ahmad , Yunzhi Huang , Lei Ma , Zhengwang Wu , Gang Li , Li Wang , Qian Wang , Pew-Thian Yap , Dinggang Shen

Brain-computer interfaces (BCIs) with speech decoding from brain recordings have broad application potential in fields such as clinical rehabilitation and cognitive neuroscience. However, current decoding methods remain limited to…

Neurons and Cognition · Quantitative Biology 2025-06-05 Yi Guo , Yihang Dong , Michael Kwok-Po Ng , Shuqiang Wang

Transformers have emerged as viable alternatives to convolutional neural networks owing to their ability to learn non-local region relationships in the spatial domain. The self-attention mechanism of the transformer enables transformers to…

Image and Video Processing · Electrical Eng. & Systems 2023-08-09 Rahul G. S. , Sriprabha Ramnarayanan , Mohammad Al Fahim , Keerthi Ram , Preejith S. P , Mohanasankar Sivaprakasam

Understanding how the brain encodes visual information is a central challenge in neuroscience and machine learning. A promising approach is to reconstruct visual stimuli, essentially images, from functional Magnetic Resonance Imaging (fMRI)…

Computer Vision and Pattern Recognition · Computer Science 2026-05-15 Zheng Huang , Enpei Zhang , Weikang Qiu , Yinghao Cai , Carl Yang , Elynn Chen , Xiang Zhang , Rex Ying , Dawei Zhou , Yujun Yan

Automatic semantic segmentation of magnetic resonance imaging (MRI) images using deep neural networks greatly assists in evaluating and planning treatments for various clinical applications. However, training these models is conditioned on…

Computer Vision and Pattern Recognition · Computer Science 2024-01-17 Navapat Nananukul , Hamid Soltanian-zadeh , Mohammad Rostami

Magnetic Resonance Imaging (MRI) is a principal diagnostic approach used in the field of radiology to create images of the anatomical and physiological structure of patients. MRI is the prevalent medical imaging practice to find…

Computer Vision and Pattern Recognition · Computer Science 2021-06-16 Yusuf Brima , Mossadek Hossain Kamal Tushar , Upama Kabir , Tariqul Islam

Human brains exhibit highly organized multiscale neurophysiological dynamics. Understanding those dynamic changes and the neuronal networks involved is critical for understanding how the brain functions in health and disease. Functional…

Neurons and Cognition · Quantitative Biology 2024-09-09 Manuel Morante , Kristian Frølich , Naveed ur Rehman

This study introduces a pioneering approach in brain-computer interface (BCI) technology, featuring our novel concept of complex visual imagery for non-invasive electroencephalography (EEG)-based communication. Complex visual imagery, as…

Human-Computer Interaction · Computer Science 2025-11-20 Byoung-Hee Kwon

The application of deep learning (DL) models to neuroimaging data poses several challenges, due to the high dimensionality, low sample size and complex temporo-spatial dependency structure of these datasets. Even further, DL models act as…

Machine Learning · Computer Science 2019-04-08 Armin W. Thomas , Hauke R. Heekeren , Klaus-Robert Müller , Wojciech Samek

Deep learning models for brain tumor analysis require large and diverse datasets that are often siloed across healthcare institutions due to privacy regulations. We present a federated learning framework for brain tumor localization that…

Computer Vision and Pattern Recognition · Computer Science 2026-01-22 Andrea Protani , Riccardo Taiello , Marc Molina Van Den Bosch , Luigi Serio

The goal of emotional brain state classification on functional MRI (fMRI) data is to recognize brain activity patterns related to specific emotion tasks performed by subjects during an experiment. Distinguishing emotional brain states from…

Image and Video Processing · Electrical Eng. & Systems 2022-11-01 Maxime Tchibozo , Donggeun Kim , Zijing Wang , Xiaofu He

Brain decoding, a pivotal field in neuroscience, aims to reconstruct stimuli from acquired brain signals, primarily utilizing functional magnetic resonance imaging (fMRI). Currently, brain decoding is confined to a per-subject-per-model…

Computer Vision and Pattern Recognition · Computer Science 2024-04-12 Shizun Wang , Songhua Liu , Zhenxiong Tan , Xinchao Wang

Magnetic Resonance Imaging (MRI) is a pivotal clinical diagnostic tool, yet its extended scanning times often compromise patient comfort and image quality, especially in volumetric, temporal and quantitative scans. This review elucidates…

Human state recognition is a critical topic with pervasive and important applications in human-machine systems. Multi-modal fusion, the combination of metrics from multiple data sources, has been shown as a sound method for improving the…

Human-Computer Interaction · Computer Science 2023-04-12 Ruiqi Wang , Wonse Jo , Dezhong Zhao , Weizheng Wang , Baijian Yang , Guohua Chen , Byung-Cheol Min

In medical image segmentation, specialized computer vision techniques, notably transformers grounded in attention mechanisms and residual networks employing skip connections, have been instrumental in advancing performance. Nonetheless,…

Computer Vision and Pattern Recognition · Computer Science 2026-05-26 Fuchen Zheng , Xuhang Chen , Weihuang Liu , Haolun Li , Yingtie Lei , Jiahui He , Chi-Man Pun , Shounjun Zhou