English
Related papers

Related papers: VoiceGiraffe: A Benchmark for Extreme Long-Context…

200 papers

Large Audio Language Models (LALM) combine the audio perception models and the Large Language Models (LLM) and show a remarkable ability to reason about the input audio, infer the meaning, and understand the intent. However, these systems…

Audio and Speech Processing · Electrical Eng. & Systems 2024-11-26 Saurabhchand Bhati , Yuan Gong , Leonid Karlinsky , Hilde Kuehne , Rogerio Feris , James Glass

Large Language Models (LLMs) represent a landmark achievement in Artificial Intelligence (AI), demonstrating unprecedented proficiency in procedural tasks such as text generation, code completion, and conversational coherence. These…

Artificial Intelligence · Computer Science 2025-05-07 Schaun Wheeler , Olivier Jeunen

The sequential structure of videos poses a challenge to the ability of multimodal large language models (MLLMs) to locate multi-frame evidence and conduct multimodal reasoning. However, existing video benchmarks mainly focus on…

Computer Vision and Pattern Recognition · Computer Science 2025-06-05 Kejian Zhu , Zhuoran Jin , Hongbang Yuan , Jiachun Li , Shangqing Tu , Pengfei Cao , Yubo Chen , Kang Liu , Jun Zhao

Large Audio Language Models (LALMs), powered by the chain-of-thought (CoT) paradigm, have shown remarkable reasoning capabilities. Intuitively, different problems often require varying depths of reasoning. While some methods can determine…

Machine Learning · Computer Science 2025-11-20 Zhichao Sheng , Shilin Zhou , Chen Gong , Zhenghua Li

Even without directly hearing sounds, humans can effortlessly reason about auditory properties, such as pitch, loudness, or sound-source associations, drawing on auditory commonsense. In contrast, language models often lack this capability,…

Computation and Language · Computer Science 2026-01-29 Hyunjong Ok , Suho Yoo , Hyeonjun Kim , Jaeho Lee

Large Language Models (LLMs) demonstrate impressive zero-shot performance across a wide range of natural language processing tasks. Integrating various modality encoders further expands their capabilities, giving rise to Multimodal Large…

Sound · Computer Science 2026-01-13 Hao Cheng , Erjia Xiao , Jing Shao , Yichi Wang , Le Yang , Chao Shen , Philip Torr , Jindong Gu , Renjing Xu

Voice assistants increasingly rely on Speech Language Models (SpeechLMs) to interpret spoken queries and execute complex tasks, yet existing benchmarks lack domain breadth, acoustic diversity, and compositional reasoning complexity to…

While large language models (LLMs) excel at many domain-specific tasks, their ability to deeply comprehend and reason about full-length academic papers remains underexplored. Existing benchmarks often fall short of capturing such depth,…

Artificial Intelligence · Computer Science 2026-01-08 Xinbang Dai , Huikang Hu , Yongrui Chen , Jiaqi Li , Rihui Jin , Yuyang Zhang , Xiaoguang Li , Lifeng Shang , Guilin Qi

Speech large language models (SpeechLLMs) have extended human-machine interactions from the text modality to the dynamic speech domain. Spoken dialogues convey diverse information, including semantic concepts, acoustic variations,…

Computation and Language · Computer Science 2026-01-14 Heyang Liu , Yuhao Wang , Ziyang Cheng , Hongcheng Liu , Yiqi Li , Yixuan Hou , Ronghua Wu , Qunshan Gu , Yanfeng Wang , Yu Wang

Although text-to-audio generation has made remarkable progress in realism and diversity, the development of evaluation metrics has not kept pace. Widely-adopted approaches, typically based on embedding similarity like CLAPScore, effectively…

Audio and Speech Processing · Electrical Eng. & Systems 2026-01-22 Chun-Yi Kuan , Kai-Wei Chang , Hung-yi Lee

Recent Large Audio Language Models (LALMs) excel in understanding but often lack transparent reasoning. To address this "black-box" limitation, we organized the Audio Reasoning Challenge at Interspeech 2026, the first shared task dedicated…

Large language models often suffer from fact loss, timeline confusion, persona drift, and reduced stability during long-range interaction, especially under high-noise knowledge bases, context clearing, and cross-model transfer. To address…

Artificial Intelligence · Computer Science 2026-05-15 Zhao Yang , Wang Huan , Li Yingshuo , Tu Haomiao , Lin Hujite

Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations with more extensive histories. While proprietary models such…

Computation and Language · Computer Science 2023-10-05 Chenxin An , Shansan Gong , Ming Zhong , Xingjian Zhao , Mukai Li , Jun Zhang , Lingpeng Kong , Xipeng Qiu

Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipulating objects that become temporarily occluded. Recent vision-language-action (VLA) models have…

Robotics · Computer Science 2026-05-27 Yinpei Dai , Hongze Fu , Jayjun Lee , Yuejiang Liu , Haoran Zhang , Jianing Yang , Chelsea Finn , Nima Fazeli , Joyce Chai

Large Audio-Language Models (LALMs) are often constrained by short audio context windows, even when their text backbones support long contexts, limiting long-form audio understanding. Prior work has introduced context-extension methods…

Computation and Language · Computer Science 2026-01-22 Yuatyong Chaichana , Pittawat Taveekitworachai , Warit Sirichotedumrong , Potsawee Manakul , Kunat Pipatanakul

We propose WHISPER-GPT: A generative large language model (LLM) for speech and music that allows us to work with continuous audio representations and discrete tokens simultaneously as part of a single architecture. There has been a huge…

Sound · Computer Science 2024-12-20 Prateek Verma

Audio fingerprinting converts audio to much lower-dimensional representations, allowing distorted recordings to still be recognized as their originals through similar fingerprints. Existing deep learning approaches rigidly fingerprint…

Sound · Computer Science 2026-03-26 Hongjie Chen , Hanyu Meng , Huimin Zeng , Ryan A. Rossi , Lie Lu , Josh Kimball

The recent surge in open-source Multimodal Large Language Models (MLLM) frameworks, such as LLaVA, provides a convenient kickoff for artificial intelligence developers and researchers. However, most of the MLLM frameworks take vision as the…

Recent advances in large audio language models (LALMs) have greatly enhanced multimodal conversational systems. However, existing benchmarks remain limited -- they are mainly English-centric, rely on synthetic speech, and lack…

Sound · Computer Science 2026-02-10 Jiliang Hu , Wenfu Wang , Zuchao Li , Chenxing Li , Yiyang Zhao , Hanzhao Li , Liqiang Zhang , Meng Yu , Dong Yu

While recent advancements in vision-language models have improved video understanding, diagnosing their capacity for deep, narrative comprehension remains a challenge. Existing benchmarks often test short-clip recognition or use…

Computer Vision and Pattern Recognition · Computer Science 2025-09-18 Nisarg A. Shah , Amir Ziai , Chaitanya Ekanadham , Vishal M. Patel