English
Related papers

Related papers: CMDAR: A Chinese Multi-scene Dynamic Audio Reasoni…

200 papers

We introduce MMAR, a new benchmark designed to evaluate the deep reasoning capabilities of Audio-Language Models (ALMs) across massive multi-disciplinary tasks. MMAR comprises 1,000 meticulously curated audio-question-answer triplets,…

The ability to comprehend audio--which includes speech, non-speech sounds, and music--is crucial for AI agents to interact effectively with the world. We present MMAU, a novel benchmark designed to evaluate multimodal audio understanding…

Audio and Speech Processing · Electrical Eng. & Systems 2024-10-28 S Sakshi , Utkarsh Tyagi , Sonal Kumar , Ashish Seth , Ramaneswaran Selvakumar , Oriol Nieto , Ramani Duraiswami , Sreyan Ghosh , Dinesh Manocha

The present benchmarks for testing the audio modality of multimodal large language models concentrate on testing various audio tasks such as speaker diarization or gender identification in isolation. Whether a multimodal model can answer…

Practical dialog systems need to deal with various knowledge sources, noisy user expressions, and the shortage of annotated data. To better solve the above problems, we propose CGoDial, new challenging and comprehensive Chinese benchmark…

Computation and Language · Computer Science 2022-11-22 Yinpei Dai , Wanwei He , Bowen Li , Yuchuan Wu , Zheng Cao , Zhongqi An , Jian Sun , Yongbin Li

Recently, instruction-following audio-language models have received broad attention for audio interaction with humans. However, the absence of pre-trained audio models capable of handling diverse audio types and tasks has hindered progress…

Audio and Speech Processing · Electrical Eng. & Systems 2023-12-22 Yunfei Chu , Jin Xu , Xiaohuan Zhou , Qian Yang , Shiliang Zhang , Zhijie Yan , Chang Zhou , Jingren Zhou

Recent advancements in multimodal reasoning have largely overlooked the audio modality. We introduce Audio-Reasoner, a large-scale audio language model for deep reasoning in audio tasks. We meticulously curated a large-scale and diverse…

Sound · Computer Science 2025-09-23 Zhifei Xie , Mingbao Lin , Zihang Liu , Pengcheng Wu , Shuicheng Yan , Chunyan Miao

Large Audio-Language Models (LALMs), such as GPT-4o, have recently unlocked audio dialogue capabilities, enabling direct spoken exchanges with humans. The potential of LALMs broadens their applicability across a wide range of practical…

Artificial Intelligence · Computer Science 2025-07-29 Kuofeng Gao , Shu-Tao Xia , Ke Xu , Philip Torr , Jindong Gu

Recent Large Audio Language Models (LALMs) excel in understanding but often lack transparent reasoning. To address this "black-box" limitation, we organized the Audio Reasoning Challenge at Interspeech 2026, the first shared task dedicated…

Spoken Dialogue Models (SDMs) have recently attracted significant attention for their ability to generate voice responses directly to users' spoken queries. Despite their increasing popularity, there exists a gap in research focused on…

Computation and Language · Computer Science 2025-10-07 Chengqian Ma , Wei Tao , Yiwen Guo

As multimodal content continues to expand at a rapid pace, audio retrieval has emerged as a key enabling technology for media search, content organization, and intelligent assistants. However, most existing benchmarks concentrate on…

Artificial Intelligence · Computer Science 2026-05-07 Honglei Zhang , Yuting Chen , Chenpeng Hu , Siyue Zhang , Yilei Shi

Recent multimodal large language models (MLLMs) such as GPT-4o and Qwen3-Omni show strong perception but struggle in multi-speaker, dialogue-centric settings that demand agentic reasoning tracking who speaks, maintaining roles, and…

We present Task 5 of the DCASE 2025 Challenge: an Audio Question Answering (AQA) benchmark spanning multiple domains of sound understanding. This task defines three QA subsets (Bioacoustics, Temporal Soundscapes, and Complex QA) to test…

Large audio-language models (LALMs) have achieved near-human performance in sentence-level transcription and emotion recognition. However, existing evaluations focus mainly on surface-level perception, leaving the capacity of models for…

Computation and Language · Computer Science 2025-08-05 Wanqi Yang , Yanda Li , Yunchao Wei , Meng Fang , Ling Chen

Reasoning has become a defining capability of modern foundation models, yet its development in the audio modality remains limited. Audio poses challenges that are distinct from those of text and vision. It is continuous, temporally dense,…

Audio and Speech Processing · Electrical Eng. & Systems 2026-05-21 Zhihan Guo , Wenqian Cui , Guan-Ting Lin , Daxin Tan , Jingyao Li , Qiyong Zheng , Dingdong Wang , Jing Xiong , Han Shi , Jiaya Jia , Irwin King

Understanding videos inherently requires reasoning over both visual and auditory information. To properly evaluate Omni-Large Language Models (Omni-LLMs), which are capable of processing multi-modal information including vision and audio,…

Multimedia · Computer Science 2026-05-15 Jianghan Chao , Jianzhang Gao , Wenhui Tan , Yuchong Sun , Ruihua Song , Liyun Ru

Medical audio data is difficult to collect due to privacy regulations and high annotation costs arising from domain expertise. Thus, existing benchmarks tend to underrepresent complex medical audio scenarios. To address this challenge, we…

Automatic speech recognition (ASR) in clinical dialogue demands robustness to full-duplex interaction, speaker overlap, and low-latency constraints, yet open benchmarks remain scarce. We present MMedFD, the first real-world Chinese…

Audio and Speech Processing · Electrical Eng. & Systems 2025-09-29 Hongzhao Chen , XiaoYang Wang , Jing Lan , Hexiao Ding , Yufeng Jiang , MingHui Yang , DanHui Xu , Jun Luo , Nga-Chun Ng , Gerald W. Y. Cheng , Yunlin Mao , Jung Sun Yoo

Recent advances in large language models (LLMs) have significantly improved text-to-speech (TTS) systems, enhancing control over speech style, naturalness, and emotional expression, which brings TTS Systems closer to human-level…

Full-duplex voice agents--systems that listen and speak simultaneously--are rapidly moving from research to production. However, existing evaluations address conversational dynamics and task completion in isolation. We introduce…

Sound · Computer Science 2026-03-17 Soham Ray , Keshav Dhandhania , Victor Barres , Karthik Narasimhan

Recent advances in large audio language models (LALMs) have greatly enhanced multimodal conversational systems. However, existing benchmarks remain limited -- they are mainly English-centric, rely on synthetic speech, and lack…

Sound · Computer Science 2026-02-10 Jiliang Hu , Wenfu Wang , Zuchao Li , Chenxing Li , Yiyang Zhao , Hanzhao Li , Liqiang Zhang , Meng Yu , Dong Yu
‹ Prev 1 2 3 10 Next ›