English

AV-EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Omni-modal LLMS with Audio-visual Cues

Multimedia 2026-05-29 v2 Sound

Abstract

Emotions conveyed through voice and face shape engagement and context in human AI interaction. Despite rapid progress in omni modal large language models, the holistic evaluation of emotional reasoning with audiovisual cues remains limited. To address this gap, we introduce AV EMO Reasoning, a benchmark designed to systematically assess emotional reasoning abilities in large language models. The framework uses a curated audiovisual corpus comprising synthetic single turn and multi turn dialogues and a real world subset, together with emotion perception and interaction reasoning metrics, to evaluate whether models can understand user emotions and produce appropriate responses. By releasing a systematic evaluation benchmark, AV EMO Reasoning offers a reproducible standard for evaluating emotion aware dialogue and advances toward more natural, adaptive human AI interaction.

Keywords

Cite

@article{arxiv.2510.07355,
  title  = {AV-EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Omni-modal LLMS with Audio-visual Cues},
  author = {Dingkun Zhou and Krish Patel and Ajay Kankipati and Akshaj Gupta and Zeyi Austin Li and Mohul Shukla and Vibhor Narang and Sara Kofman and Zongli Ye and Grace Wang and Xiaoyu Shi and Tingle Li and Guan-Ting Lin and Kan Jen Cheng and Huang-Cheng Chou and Jiachen Lian and Gopala Anumanchipalli},
  journal= {arXiv preprint arXiv:2510.07355},
  year   = {2026}
}