English
Related papers

Related papers: LibEMER: A novel benchmark and algorithms library …

200 papers

We proposed the industry level deep learning approach for speech emotion recognition task. In industry, carefully proposed deep transfer learning technology shows real results due to mostly low amount of training data availability, machine…

Sound · Computer Science 2021-09-10 Enkhtogtokh Togootogtokh , Christian Klasen

Large language models (LLMs) have made significant progress in Emotional Intelligence (EI) and long-context modeling. However, existing benchmarks often overlook the fact that emotional information processing unfolds as a continuous…

Computation and Language · Computer Science 2026-01-13 Weichu Liu , Jing Xiong , Yuxuan Hu , Zixuan Li , Minghuan Tan , Ningning Mao , Hui Shen , Wendong Xu , Chaofan Tao , Min Yang , Chengming Li , Lingpeng Kong , Ngai Wong

Despite their strong performance in multimodal emotion reasoning, existing Multimodal Large Language Models (MLLMs) often overlook the scenarios involving emotion conflicts, where emotional cues from different modalities are inconsistent.…

Artificial Intelligence · Computer Science 2025-10-14 Zhiyuan Han , Beier Zhu , Yanlong Xu , Peipei Song , Xun Yang

The ability to organically reason over and with both text and images is a pillar of human intelligence, yet the ability of Multimodal Large Language Models (MLLMs) to perform such multimodal reasoning remains under-explored. Existing…

Computer Vision and Pattern Recognition · Computer Science 2025-01-10 Yunzhuo Hao , Jiawei Gu , Huichen Will Wang , Linjie Li , Zhengyuan Yang , Lijuan Wang , Yu Cheng

Multimodal emotion analysis is shifting from static classification to generative reasoning. Beyond simple label prediction, robust affective reasoning must synthesize fine-grained signals such as facial micro-expressions and prosodic which…

Multimedia · Computer Science 2026-02-05 Zhixian Zhao , Wenjie Tian , Lei Xie

Recent advances in Large Language Models (LLMs) have highlighted the need for robust, comprehensive, and challenging benchmarks. Yet, research on evaluating their Emotional Intelligence (EI) is considerably limited. Existing benchmarks have…

Computation and Language · Computer Science 2024-07-18 Sahand Sabour , Siyang Liu , Zheyuan Zhang , June M. Liu , Jinfeng Zhou , Alvionna S. Sunaryo , Juanzi Li , Tatia M. C. Lee , Rada Mihalcea , Minlie Huang

Multimodal emotion recognition is a task of great concern. However, traditional data sets are based on fixed labels, resulting in models that often focus on main emotions and ignore detailed emotional changes in complex scenes. This report…

Computer Vision and Pattern Recognition · Computer Science 2024-08-23 Mengying Ge , Dongkai Tang , Mingyang Li

While text-based emotion recognition methods have achieved notable success, real-world dialogue systems often demand a more nuanced emotional understanding than any single modality can offer. Multimodal Emotion Recognition in Conversations…

Computation and Language · Computer Science 2025-09-10 Chengyan Wu , Yiqiang Cai , Yang Liu , Pengxu Zhu , Yun Xue , Ziwei Gong , Julia Hirschberg , Bolei Ma

Incomplete multi-modal emotion recognition (IMER) aims at understanding human intentions and sentiments by comprehensively exploring the partially observed multi-source data. Although the multi-modal data is expected to provide more…

Computer Vision and Pattern Recognition · Computer Science 2025-12-30 Wen-Jue He , Xiaofeng Zhu , Zheng Zhang

Multimodal affective computing has gained increasing attention due to its broad applications in understanding human behavior and intentions, particularly in text-centric multimodal scenarios. Existing research spans diverse tasks,…

Computation and Language · Computer Science 2026-04-08 Guimin Hu , Weimin Lyu , Chang Sun , Zhihong Zhu , Lin Gui , Ruichu Cai , Erik Cambria , Hasti Seifi

Emotion recognition is a topic of significant interest in assistive robotics due to the need to equip robots with the ability to comprehend human behavior, facilitating their effective interaction in our society. Consequently, efficient and…

Human-Computer Interaction · Computer Science 2023-12-05 Rutherford Agbeshi Patamia , Paulo E. Santos , Kingsley Nketia Acheampong , Favour Ekong , Kwabena Sarpong , She Kun

Automatic emotion recognition is an active research topic with wide range of applications. Due to the high manual annotation cost and inevitable label ambiguity, the development of emotion recognition dataset is limited in both scale and…

Audio and Speech Processing · Electrical Eng. & Systems 2020-09-08 Jingjun Liang , Ruichen Li , Qin Jin

Emotion recognition is involved in several real-world applications. With an increase in available modalities, automatic understanding of emotions is being performed more accurately. The success in Multimodal Emotion Recognition (MER),…

Computer Vision and Pattern Recognition · Computer Science 2022-07-26 Riccardo Franceschini , Enrico Fini , Cigdem Beyan , Alessandro Conti , Federica Arrigoni , Elisa Ricci

The furnishing of multi-modal large language models (MLLMs) has led to the emergence of numerous benchmark studies, particularly those evaluating their perception and understanding capabilities. Among these, understanding image-evoked…

Multimedia · Computer Science 2025-09-18 Lancheng Gao , Ziheng Jia , Yunhao Zeng , Wei Sun , Yiming Zhang , Wei Zhou , Guangtao Zhai , Xiongkuo Min

Emotion recognition is essential for applications in affective computing and behavioral prediction, but conventional systems relying on single-modality data often fail to capture the complexity of affective states. To address this…

Multimedia · Computer Science 2025-09-08 Jianlu Wang , Yanan Wang , Tong Liu

Multimodal Large Language Models (MLLMs) have revolutionized numerous research fields, including computer vision and affective computing. As a pivotal challenge in this interdisciplinary domain, facial expression recognition (FER) has…

Computer Vision and Pattern Recognition · Computer Science 2025-11-04 Fan Zhang , Haoxuan Li , Shengju Qian , Xin Wang , Zheng Lian , Hao Wu , Zhihong Zhu , Yuan Gao , Qiankun Li , Yefeng Zheng , Zhouchen Lin , Pheng-Ann Heng

Large Language Models (LLMs) have shown strong potential as conversational agents. Yet, their effectiveness remains limited by deficiencies in robust long-term memory, particularly in complex, long-term web-based services such as online…

Computation and Language · Computer Science 2026-02-03 Tiantian Chen , Jiaqi Lu , Ying Shen , Lin Zhang

Emotion understanding is a critical yet challenging task. Recent advances in Multimodal Large Language Models (MLLMs) have significantly enhanced their capabilities in this area. However, MLLMs often suffer from hallucinations, generating…

Computer Vision and Pattern Recognition · Computer Science 2025-05-19 Bohao Xing , Xin Liu , Guoying Zhao , Chengyu Liu , Xiaolan Fu , Heikki Kälviäinen

In today's world, emotional support is increasingly essential, yet it remains challenging for both those seeking help and those offering it. Multimodal approaches to emotional support show great promise by integrating diverse data sources…

Multimodal Emotion Recognition (MER) aims to automatically identify and understand human emotional states by integrating information from various modalities. However, the scarcity of annotated multimodal data significantly hinders the…

Human-Computer Interaction · Computer Science 2024-09-11 Zhixian Zhao , Haifeng Chen , Xi Li , Dongmei Jiang , Lei Xie