MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
Abstract
We present MERaLiON-SER, a robust speech emotion recognition model designed for English and Southeast Asian languages. The model is trained using a hybrid objective combining weighted categorical cross-entropy and Concordance Correlation Coefficient (CCC) losses for joint discrete and dimensional emotion modelling. This dual approach enables the model to capture both the distinct categories of emotion (like happy or angry) and the fine-grained, such as arousal (intensity), valence (positivity/negativity), and dominance (sense of control), leading to a more comprehensive and robust representation of human affect. Extensive evaluations across multilingual Singaporean languages (English, Chinese, Malay, and Tamil ) and other public benchmarks show that MERaLiON-SER consistently surpasses both open-source speech encoders and large Audio-LLMs. These results underscore the importance of specialised speech-only models for accurate paralinguistic understanding and cross-lingual generalisation. Furthermore, the proposed framework provides a foundation for integrating emotion-aware perception into future agentic audio systems, enabling more empathetic and contextually adaptive multimodal reasoning.
Keywords
Cite
@article{arxiv.2511.04914,
title = {MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages},
author = {Hardik B. Sailor and Aw Ai Ti and Chen Fang Yih Nancy and Chiu Ying Lay and Ding Yang and He Yingxu and Jiang Ridong and Li Jingtao and Liao Jingyi and Liu Zhuohan and Lu Yanfeng and Ma Yi and Manas Gupta and Muhammad Huzaifah Bin Md Shahrin and Nabilah Binte Md Johan and Nattadaporn Lertcheva and Pan Chunlei and Pham Minh Duc and Siti Maryam Binte Ahmad Subaidi and Siti Umairah Binte Mohammad Salleh and Sun Shuo and Tarun Kumar Vangani and Wang Qiongqiong and Won Cheng Yi Lewis and Wong Heng Meng Jeremy and Wu Jinyang and Zhang Huayun and Zhang Longyin and Zou Xunlong},
journal= {arXiv preprint arXiv:2511.04914},
year = {2025}
}
Comments
https://huggingface.co/MERaLiON/MERaLiON-SER-v1