English
Related papers

Related papers: Multimodal Speech Emotion Recognition Using Audio …

200 papers

The integration of information across multiple modalities and across time is a promising way to enhance the emotion recognition performance of affective systems. Much previous work has focused on instantaneous emotion recognition. The 2018…

Image and Video Processing · Electrical Eng. & Systems 2018-05-07 Didan Deng , Yuqian Zhou , Jimin Pi , Bertram E. Shi

Speech emotion recognition aims to identify emotional states from speech signals and has been widely applied in human-computer interaction, education, healthcare, and many other fields. However, since speech data contain rich sensitive…

Sound · Computer Science 2025-12-23 Zhao Ren , Rathi Adarshi Rammohan , Kevin Scheck , Tanja Schultz

Data-driven models achieve successful results in Speech Emotion Recognition (SER). However, these models, which are often based on general acoustic features or end-to-end approaches, show poor performance when the testing set has a…

Audio and Speech Processing · Electrical Eng. & Systems 2025-12-15 Duowei Tang , Peter Kuppens , Lucca Geurts , Toon van Waterschoot

Intelligent monitoring systems and affective computing applications have emerged in recent years to enhance healthcare. Examples of these applications include assessment of affective states such as Major Depressive Disorder (MDD). MDD…

Human-Computer Interaction · Computer Science 2020-11-19 Alice Othmani , Daoud Kadoch , Kamil Bentounes , Emna Rejaibi , Romain Alfred , Abdenour Hadid

There is an imminent need for guidelines and standard test sets to allow direct and fair comparisons of speech emotion recognition (SER). While resources, such as the Interactive Emotional Dyadic Motion Capture (IEMOCAP) database, have…

Emotion recognition based on Electroencephalography (EEG) has gained significant attention and diversified development in fields such as neural signal processing and affective computing. However, the unique brain anatomy of individuals…

Signal Processing · Electrical Eng. & Systems 2024-05-31 Yihang Dong , Xuhang Chen , Yanyan Shen , Michael Kwok-Po Ng , Tao Qian , Shuqiang Wang

Speech is the most natural way of expressing ourselves as humans. Identifying emotion from speech is a nontrivial task due to the ambiguous definition of emotion itself. Speaker Emotion Recognition (SER) is essential for understanding human…

Sound · Computer Science 2024-11-07 Pourya Jafarzadeh , Amir Mohammad Rostami , Padideh Choobdar

One of the most significant issues as attended a lot in recent years is that of recognizing the sentiments and emotions in social media texts. The analysis of sentiments and emotions is intended to recognize the conceptual information such…

Machine Learning · Computer Science 2025-03-27 Bahareh Golchin , Noushin Riahi

Emotion detection in dialogues is challenging as it often requires the identification of thematic topics underlying a conversation, the relevant commonsense knowledge, and the intricate transition patterns between the affective states. In…

Computation and Language · Computer Science 2021-06-03 Lixing Zhu , Gabriele Pergola , Lin Gui , Deyu Zhou , Yulan He

Emotion recognition in multi-speaker conversations faces significant challenges due to speaker ambiguity and severe class imbalance. We propose a novel framework that addresses these issues through three key innovations: (1) a speaker…

Sound · Computer Science 2025-11-19 Xiao Li , Kotaro Funakoshi , Manabu Okumura

Deep learning approaches for sentiment classification do not fully exploit sentiment linguistic knowledge. In this paper, we propose a Multi-sentiment-resource Enhanced Attention Network (MEAN) to alleviate the problem by integrating three…

Computation and Language · Computer Science 2018-07-16 Zeyang Lei , Yujiu Yang , Min Yang , Yi Liu

Emotion recognition is inherently ambiguous, with uncertainty arising both from rater disagreement and from discrepancies across modalities such as speech and text. There is growing interest in modeling rater ambiguity using label…

Audio and Speech Processing · Electrical Eng. & Systems 2026-01-27 Jingyao Wu , Grace Lin , Yinuo Song , Rosalind Picard

We propose a nonparallel data-driven emotional speech conversion method. It enables the transfer of emotion-related characteristics of a speech signal while preserving the speaker's identity and linguistic content. Most existing approaches…

Machine Learning · Computer Science 2020-04-14 Jian Gao , Deep Chakraborty , Hamidou Tembine , Olaitan Olaleye

One of the key points in music recommendation is authoring engaging playlists according to sentiment and emotions. While previous works were mostly based on audio for music discovery and playlists generation, we take advantage of our…

Computation and Language · Computer Science 2019-01-16 Loreto Parisi , Simone Francia , Silvio Olivastri , Maria Stella Tavella

We investigate the effect and usefulness of spontaneity (i.e. whether a given speech is spontaneous or not) in speech in the context of emotion recognition. We hypothesize that emotional content in speech is interrelated with its…

Audio and Speech Processing · Electrical Eng. & Systems 2018-06-15 Karttikeya Mangalam , Tanaya Guha

Accurately detecting emotions in conversation is a necessary yet challenging task due to the complexity of emotions and dynamics in dialogues. The emotional state of a speaker can be influenced by many different factors, such as…

Computation and Language · Computer Science 2023-02-07 Jiachen Luo , Huy Phan , Joshua Reiss

Large speech emotion recognition datasets are hard to obtain, and small datasets may contain biases. Deep-net-based classifiers, in turn, are prone to exploit those biases and find shortcuts such as speaker characteristics. These shortcuts…

Machine Learning · Computer Science 2022-11-08 Itai Gat , Hagai Aronowitz , Weizhong Zhu , Edmilson Morais , Ron Hoory

Non-speech emotion recognition has a wide range of applications including healthcare, crime control and rescue, and entertainment, to name a few. Providing these applications using edge computing has great potential, however, recent studies…

Sound · Computer Science 2023-05-02 Ibrahim Malik , Siddique Latif , Sanaullah Manzoor , Muhammad Usama , Junaid Qadir , Raja Jurdak

Emotion recognition and classification is a very active area of research. In this paper, we present a first approach to emotion classification using persistent entropy and support vector machines. A topology-based model is applied to obtain…

Sound · Computer Science 2019-03-22 R. Gonzalez-Diaz , E. Paluzo-Hidalgo , J. F. Quesada

We propose a framework for multimodal sentiment analysis and emotion recognition using convolutional neural network-based feature extraction from text and visual modalities. We obtain a performance improvement of 10% over the state of the…

Multimedia · Computer Science 2017-08-01 Erik Cambria , Devamanyu Hazarika , Soujanya Poria , Amir Hussain , R. B. V. Subramaanyam