English
Related papers

Related papers: Toward Robust Incomplete Multimodal Sentiment Anal…

200 papers

Multi-modal emotion recognition in conversations is a challenging problem due to the complex and complementary interactions between different modalities. Audio and textual cues are particularly important for understanding emotions from a…

Sound · Computer Science 2025-04-02 Jiachen Luo , Huy Phan , Lin Wang , Joshua Reiss

With the rapid development of the internet, the richness of User-Generated Contentcontinues to increase, making Multimodal Aspect-Based Sentiment Analysis (MABSA) a research hotspot. Existing studies have achieved certain results in MABSA,…

Artificial Intelligence · Computer Science 2024-10-21 Xiaoyong Huang , Heli Sun , Qunshu Gao , Wenjie Huang , Ruichen Cao

Multimodal Aspect-Based Sentiment Analysis (MABSA) aims to extract aspect terms and their corresponding sentiment polarities from multimodal information, including text and images. While traditional supervised learning methods have shown…

Computation and Language · Computer Science 2024-11-26 Shezheng Song

Multimodal speech emotion recognition (SER) has emerged as pivotal for improving human-machine interaction. Researchers are increasingly leveraging both speech and textual information obtained through automatic speech recognition (ASR) to…

Human-Computer Interaction · Computer Science 2025-09-24 Jiajun He , Xiaohan Shi , Cheng-Hung Hu , Jinyi Mi , Xingfeng Li , Tomoki Toda

Multimodal sentiment analysis, a pivotal task in affective computing, seeks to understand human emotions by integrating cues from language, audio, and visual signals. While many recent approaches leverage complex attention mechanisms and…

Computation and Language · Computer Science 2025-05-09 Nischal Mandal , Yang Li

As a vital aspect of affective computing, Multimodal Emotion Recognition has been an active research area in the multimedia community. Despite recent progress, this field still confronts two major challenges in real-world applications: 1)…

Computer Vision and Pattern Recognition · Computer Science 2023-12-27 Chengxin Chen , Pengyuan Zhang

In this paper, we propose a new Multimodal Representation Learning (MRL) method for Multimodal Sentiment Analysis (MSA), which facilitates the adaptive interaction between modalities through Cooperative Sentiment Agents, named Co-SA. Co-SA…

Computation and Language · Computer Science 2024-04-22 Shanmin Wang , Hui Shuai , Qingshan Liu , Fei Wang

Multimodal Sentiment Analysis (MSA) is a rapidly developing field that integrates multimodal information to recognize sentiments, and existing models have made significant progress in this area. The central challenge in MSA is multimodal…

Computation and Language · Computer Science 2025-08-25 Yijie Jin , Junjie Peng , Xuanchao Lin , Haochen Yuan , Lan Wang , Cangzhi Zheng

Developing effective multimodal data fusion strategies has become increasingly essential for improving the predictive power of statistical machine learning methods across a wide range of applications, from autonomous driving to medical…

Machine Learning · Computer Science 2025-07-29 Ziyi Liang , Annie Qu , Babak Shahbaba

Multimodal Sentiment Analysis (MSA) requires integrating language, acoustic, and visual signals without sacrificing modality-specific sentiment evidence. Existing methods mainly improve either shared-private decomposition or cross-modal…

Multimedia · Computer Science 2026-04-29 Chunlei Meng , Jiabin Luo , Pengbin Feng , Zhenglin Yan , Chengyin Hu , Zhongxue Gan , Chun Ouyang

Multi-modal Emotion Recognition in Conversation (MERC) has received considerable attention in various fields, e.g., human-computer interaction and recommendation systems. Most existing works perform feature disentanglement and fusion to…

Computation and Language · Computer Science 2024-05-06 Yuntao Shou , Tao Meng , Fuchen Zhang , Nan Yin , Keqin Li

Multimodal Sentiment Analysis (MSA) has been a popular topic in natural language processing nowadays, at both sentence and aspect level. However, the existing approaches almost require large-size labeled datasets, which bring about large…

Computation and Language · Computer Science 2023-08-01 Zikai Zhou , Haisong Feng , Baiyou Qiao , Gang Wu , Donghong Han

Multimodal Machine Learning has emerged as a prominent research direction across various applications such as Sentiment Analysis, Emotion Recognition, Machine Translation, Hate Speech Recognition, and Movie Genre Classification. This…

Computation and Language · Computer Science 2023-06-13 Abdelhamid Haouhat , Slimane Bellaouar , Attia Nehar , Hadda Cherroun

Multimodal sentiment analysis (MSA) leverages information fusion from diverse modalities (e.g., text, audio, visual) to enhance sentiment prediction. However, simple fusion techniques often fail to account for variations in modality…

Machine Learning · Computer Science 2025-10-03 Han Wu , Yanming Sun , Yunhe Yang , Derek F. Wong

Multimodal Sentiment Analysis (MSA) aims to understand human intentions by integrating emotion-related clues from diverse modalities, such as visual, language, and audio. Unfortunately, the current MSA task invariably suffers from unplanned…

Computation and Language · Computer Science 2024-07-08 Dingkang Yang , Mingcheng Li , Dongling Xiao , Yang Liu , Kun Yang , Zhaoyu Chen , Yuzheng Wang , Peng Zhai , Ke Li , Lihua Zhang

The development of multimodal models has significantly advanced multimodal sentiment analysis and emotion recognition. However, in real-world applications, the presence of various missing modality cases often leads to a degradation in the…

Computation and Language · Computer Science 2024-07-09 Zirun Guo , Tao Jin , Zhou Zhao

Multimodal emotion recognition (MER) is crucial for human-computer interaction, yet real-world challenges like dynamic modality incompleteness and asynchrony severely limit its robustness. Existing methods often assume consistently complete…

Human-Computer Interaction · Computer Science 2025-08-19 Yitong Zhu , Lei Han , Guanxuan Jiang , PengYuan Zhou , Yuyang Wang

Multimodal affective computing has gained increasing attention due to its broad applications in understanding human behavior and intentions, particularly in text-centric multimodal scenarios. Existing research spans diverse tasks,…

Computation and Language · Computer Science 2026-04-08 Guimin Hu , Weimin Lyu , Chang Sun , Zhihong Zhu , Lin Gui , Ruichu Cai , Erik Cambria , Hasti Seifi

Multimodal sentiment analysis (MSA) aims to understand human emotions by integrating information from multiple modalities, such as text, audio, and visual data. However, existing methods often suffer from spurious correlations both within…

Machine Learning · Computer Science 2026-05-21 Menghua Jiang , Yuxia Lin , Baoliang Chen , Haifeng Hu , Yuncheng Jiang , Sijie Mai

Multimodal fusion is considered a key step in multimodal tasks such as sentiment analysis, emotion detection, question answering, and others. Most of the recent work on multimodal fusion does not guarantee the fidelity of the multimodal…

Machine Learning · Computer Science 2019-08-19 Navonil Majumder , Soujanya Poria , Gangeshwar Krishnamurthy , Niyati Chhaya , Rada Mihalcea , Alexander Gelbukh