English
Related papers

Related papers: AffectGPT: Dataset and Framework for Explainable M…

200 papers

A prevalent shortfall among current empathic AI systems is their inability to recognize when verbal expressions may not fully reflect underlying emotional states. This is because the existing datasets, used for the training of these…

Human-Computer Interaction · Computer Science 2025-09-09 Sharjeel Tahir , Judith Johnson , Jumana Abu-Khalaf , Syed Afaq Ali Shah

Emotion recognition and understanding is a vital component in human-machine interaction. Dimensional models of affect such as those using valence and arousal have advantages over traditional categorical ones due to the complexity of…

Computer Vision and Pattern Recognition · Computer Science 2021-09-01 Vassilios Vonikakis , Dexter Neo , Stefan Winkler

One of the most significant challenges in Music Emotion Recognition (MER) comes from the fact that emotion labels can be heterogeneous across datasets with regard to the emotion representation, including categorical (e.g., happy, sad)…

Sound · Computer Science 2025-04-14 Jaeyong Kang , Dorien Herremans

The integration of information across multiple modalities and across time is a promising way to enhance the emotion recognition performance of affective systems. Much previous work has focused on instantaneous emotion recognition. The 2018…

Image and Video Processing · Electrical Eng. & Systems 2018-05-07 Didan Deng , Yuqian Zhou , Jimin Pi , Bertram E. Shi

This paper delves into enhancing the classification performance on the GoEmotions dataset, a large, manually annotated dataset for emotion detection in text. The primary goal of this paper is to address the challenges of detecting subtle…

Computation and Language · Computer Science 2024-04-10 Kaipeng Wang , Zhi Jing , Yongye Su , Yikun Han

Computational modeling of the emotions evoked by art in humans is a challenging problem because of the subjective and nuanced nature of art and affective signals. In this paper, we consider the above-mentioned problem of understanding…

Computer Vision and Pattern Recognition · Computer Science 2021-10-14 Digbalay Bose , Krishna Somandepalli , Souvik Kundu , Rimita Lahiri , Jonathan Gratch , Shrikanth Narayanan

Foundation models have shown great promise in speech emotion recognition (SER) by leveraging their pre-trained representations to capture emotion patterns in speech signals. To further enhance SER performance across various languages and…

Computation and Language · Computer Science 2024-06-18 Shahin Amiriparian , Filip Packań , Maurice Gerczuk , Björn W. Schuller

In this work, we present a lightweight and privacy-preserving Multimodal Emotion Recognition (MER) framework designed for deployment on edge devices. To demonstrate framework's versatility, our implementation uses three modalities - speech,…

Artificial Intelligence · Computer Science 2026-02-11 Rémi Grzeczkowicz , Eric Soriano , Ali Janati , Miyu Zhang , Gerard Comas-Quiles , Victor Carballo Araruna , Aneesh Jonelagadda

Multimodal emotion recognition (MER), leveraging speech and text, has emerged as a pivotal domain within human-computer interaction, demanding sophisticated methods for effective multimodal integration. The challenge of aligning features…

Audio and Speech Processing · Electrical Eng. & Systems 2024-12-31 Xuechen Wang , Shiwan Zhao , Haoqin Sun , Hui Wang , Jiaming Zhou , Yong Qin

Multimodal emotion recognition plays a crucial role in enhancing user experience in human-computer interaction. Over the past few decades, researchers have proposed a series of algorithms and achieved impressive progress. Although each…

Human-Computer Interaction · Computer Science 2024-04-23 Zheng Lian , Licai Sun , Yong Ren , Hao Gu , Haiyang Sun , Lan Chen , Bin Liu , Jianhua Tao

Multimodal emotion recognition (MER) aims to identify human emotions by combining data from various modalities such as language, audio, and vision. Despite the recent advances of MER approaches, the limitations in obtaining extensive…

Computer Vision and Pattern Recognition · Computer Science 2025-06-03 Yehun Song , Sunyoung Cho

Emotion recognition is essential for applications in affective computing and behavioral prediction, but conventional systems relying on single-modality data often fail to capture the complexity of affective states. To address this…

Multimedia · Computer Science 2025-09-08 Jianlu Wang , Yanan Wang , Tong Liu

Emotion recognition is a topic of significant interest in assistive robotics due to the need to equip robots with the ability to comprehend human behavior, facilitating their effective interaction in our society. Consequently, efficient and…

Human-Computer Interaction · Computer Science 2023-12-05 Rutherford Agbeshi Patamia , Paulo E. Santos , Kingsley Nketia Acheampong , Favour Ekong , Kwabena Sarpong , She Kun

Speech emotion recognition (SER) systems aim to recognize human emotional state during human-computer interaction. Most existing SER systems are trained based on utterance-level labels. However, not all frames in an audio have affective…

Sound · Computer Science 2023-12-29 Qifei Li , Yingming Gao , Cong Wang , Yayue Deng , Jinlong Xue , Yichen Han , Ya Li

Understanding emotion expressed in language has a wide range of applications, from building empathetic chatbots to detecting harmful online behavior. Advancement in this area can be improved using large-scale datasets with a fine-grained…

Computation and Language · Computer Science 2020-06-04 Dorottya Demszky , Dana Movshovitz-Attias , Jeongwoo Ko , Alan Cowen , Gaurav Nemade , Sujith Ravi

While increasing research focuses on the emotional well-being of agile team members, a significant gap remains in emotion monitoring studies for Scrum Masters and meeting organizers, whose impact on team dynamics is crucial. This paper…

Artificial Intelligence · Computer Science 2026-05-19 Jingni Huang , Peter Bloodsworth

Emotion recognition based on Electroencephalography (EEG) has gained significant attention and diversified development in fields such as neural signal processing and affective computing. However, the unique brain anatomy of individuals…

Signal Processing · Electrical Eng. & Systems 2024-05-31 Yihang Dong , Xuhang Chen , Yanyan Shen , Michael Kwok-Po Ng , Tao Qian , Shuqiang Wang

Emotion recognition in text, the task of identifying emotions such as joy or anger, is a challenging problem in NLP with many applications. One of the challenges is the shortage of available datasets that have been annotated with emotions.…

Computation and Language · Computer Science 2023-10-31 Anna Koufakou , Diego Grisales , Ragy Costa de jesus , Oscar Fox

Foundation models have shown superior performance for speech emotion recognition (SER). However, given the limited data in emotion corpora, finetuning all parameters of large pre-trained models for SER can be both resource-intensive and…

Audio and Speech Processing · Electrical Eng. & Systems 2024-04-02 Nineli Lashkarashvili , Wen Wu , Guangzhi Sun , Philip C. Woodland

Training emotion recognition models has relied heavily on human annotated data, which present diversity, quality, and cost challenges. In this paper, we explore the potential of Large Language Models (LLMs), specifically GPT4, in automating…

Computation and Language · Computer Science 2024-09-02 Minxue Niu , Mimansa Jaiswal , Emily Mower Provost