English
Related papers

Related papers: Knowledge-enhanced Multimodal ECG Representation L…

200 papers

Electroencephalography (EEG) is a fundamental modality for cognitive state monitoring in brain-computer interfaces (BCIs). However, it is highly susceptible to intrinsic signal errors and human-induced labeling errors, which lead to label…

Machine Learning · Computer Science 2025-12-15 Hyo-Jeong Jang , Hye-Bin Shin , Seong-Whan Lee

Emotional Recognition in Conversation (ERC) is valuable for diagnosing health conditions such as autism and depression, and for understanding the emotions of individuals who struggle to express their feelings. Current ERC methods primarily…

Human-Computer Interaction · Computer Science 2026-05-06 Zijian Kang , Yueyang Li , Shengyu Gong , Weiming Zeng , Hongjie Yan , Lingbin Bian , Zhiguo Zhang , Wai Ting Siok , Nizhuan Wang

The performances of commonly used electrocardiogram (ECG) diagnosis models have recently improved with the introduction of deep learning (DL). However, the impact of various combinations of multiple DL components and/or the role of data…

Signal Processing · Electrical Eng. & Systems 2022-08-02 Jae-Won Choi , Dae-Yong Hong , Chan Jung , Eugene Hwang , Sung-Hyuk Park , Seung-Young Roh

The electrocardiogram (ECG) is one of the most commonly used non-invasive, convenient medical monitoring tools that assist in the clinical diagnosis of heart diseases. Recently, deep learning (DL) techniques, particularly self-supervised…

Machine Learning · Computer Science 2023-03-23 Jun Li , Che Liu , Sibo Cheng , Rossella Arcucci , Shenda Hong

Clinical 12-lead electrocardiography (ECG) is one of the most widely encountered kinds of biosignals. Despite the increased availability of public ECG datasets, label scarcity remains a central challenge in the field. Self-supervised…

Signal Processing · Electrical Eng. & Systems 2022-01-05 Temesgen Mehari , Nils Strodthoff

Integrating physiological signals such as electroencephalogram (EEG), with other data such as interview audio, may offer valuable multimodal insights into psychological states or neurological disorders. Recent advancements with Large…

Human-Computer Interaction · Computer Science 2024-08-15 Yongquan Hu , Shuning Zhang , Ting Dang , Hong Jia , Flora D. Salim , Wen Hu , Aaron J. Quigley

Recent advances in large language models (LLMs) have enabled the development of multimodal medical AI. While models such as MedGemini achieve high accuracy on VQA tasks like USMLE MM, their performance on ECG based tasks remains limited,…

Machine Learning · Computer Science 2026-02-12 Junichiro Takahashi , Masataka Sato , Satoshi Kodeta , Norihiko Takeda

Detecting Alzheimer's Disease (AD) from narrative transcripts remains a challenging task for large language models (LLMs), particularly under out-of-distribution (OOD) and data-scarce conditions. While in-context learning (ICL) provides a…

Computation and Language · Computer Science 2025-11-11 Puzhen Su , Yongzhu Miao , Chunxi Guo , Jintao Tang , Shasha Li , Ting Wang

Continuous electroencephalography (EEG) is routinely used in neurocritical care to monitor seizures and other harmful brain activity, including rhythmic and periodic patterns that are clinically significant. Although deep learning methods…

Human-Computer Interaction · Computer Science 2026-01-05 Argha Kamal Samanta , Deepak Mewada , Monalisa Sarma , Debasis Samanta

Multimodal emotion recognition (MER) aims to detect the emotional status of a given expression by combining the speech and text information. Intuitively, label information should be capable of helping the model locate the salient…

Computation and Language · Computer Science 2023-09-06 Peiying Wang , Sunlu Zeng , Junqing Chen , Lu Fan , Meng Chen , Youzheng Wu , Xiaodong He

While recent multimodal large language models (MLLMs) have advanced automated ECG interpretation, they still face two key limitations: (1) insufficient multimodal synergy between time series signals and visual ECG representations, and (2)…

Computation and Language · Computer Science 2025-10-21 Xiang Lan , Feng Wu , Kai He , Qinghao Zhao , Shenda Hong , Mengling Feng

Relational learning is an essential task in the domain of knowledge representation, particularly in knowledge graph completion (KGC). While relational learning in traditional single-modal settings has been extensively studied, exploring it…

Machine Learning · Computer Science 2024-12-05 Rui Cai , Shichao Pei , Xiangliang Zhang

In the medical field, current ECG signal analysis approaches rely on supervised deep neural networks trained for specific tasks that require substantial amounts of labeled data. However, our paper introduces ECGBERT, a self-supervised…

Signal Processing · Electrical Eng. & Systems 2023-06-13 Seokmin Choi , Sajad Mousavi , Phillip Si , Haben G. Yhdego , Fatemeh Khadem , Fatemeh Afghah

Electrocardiograms (ECGs) are among the most widely used diagnostic tools for cardiovascular diseases, and a large amount of ECG data worldwide appears only in image form. However, most existing automated ECG analysis methods rely on access…

Machine Learning · Computer Science 2026-04-03 Hung Manh Pham , Jialu Tang , Aaqib Saeed , Dong Ma , Bin Zhu , Pan Zhou

The electrocardiogram (ECG) is a fundamental tool in cardiovascular diagnostics due to its powerful and non-invasive nature. One of the most critical usages is to determine whether more detailed examinations are necessary, with users…

Artificial Intelligence · Computer Science 2025-04-15 Junichiro Takahashi , JingChuan Guan , Masataka Sato , Kaito Baba , Kazuto Haruguchi , Daichi Nagashima , Satoshi Kodera , Norihiko Takeda

Automated interpretation of electrocardiograms (ECG) has garnered significant attention with the advancements in machine learning methodologies. Despite the growing interest, most current studies focus solely on classification or regression…

Signal Processing · Electrical Eng. & Systems 2023-11-07 Jielin Qiu , Jiacheng Zhu , Shiqi Liu , William Han , Jingqi Zhang , Chaojing Duan , Michael Rosenberg , Emerson Liu , Douglas Weber , Ding Zhao

The main task of Multimodal Emotion Recognition in Conversations (MERC) is to identify the emotions in modalities, e.g., text, audio, image and video, which is a significant development direction for realizing machine intelligence. However,…

Sound · Computer Science 2023-12-12 Tao Meng , Yuntao Shou , Wei Ai , Nan Yin , Keqin Li

Electrocardiogram (ECG) signal is one of the most effective sources of information mainly employed for the diagnosis and prediction of cardiovascular diseases (CVDs) connected with the abnormalities in heart rhythm. Clearly, single modality…

Signal Processing · Electrical Eng. & Systems 2022-10-13 Thinh Phan , Duc Le , Patel Brijesh , Donald Adjeroh , Jingxian Wu , Morten Olgaard Jensen , Ngan Le

This position paper argues that current benchmarking practice in 12-lead ECG representation learning must be fixed to ensure progress is reliable and aligned with clinically meaningful objectives. The field has largely converged on three…

Machine Learning · Computer Science 2026-02-20 Zachary Berger , Daniel Prakah-Asante , John Guttag , Collin M. Stultz

Large language models (LLMs) enable waveform-to-text ECG interpretation and interactive clinical questioning, yet most ECG-LLM systems still rely on weak signal-text alignment and retrieval without explicit physiological or causal…