English
Related papers

Related papers: How Far Are Video Models from True Multimodal Reas…

200 papers

Rapid advances in multimodal models demand benchmarks that rigorously evaluate understanding and reasoning in safety-critical, dynamic real-world settings. We present AccidentBench, a large-scale benchmark that combines vehicle accident…

Recent advancements in Large Language Models (LLMs) have led to the development of Video Large Multi-modal Models (Video-LMMs) that can handle a wide range of video understanding tasks. These models have the potential to be deployed in…

Computer Vision and Pattern Recognition · Computer Science 2024-05-10 Muhammad Uzair Khattak , Muhammad Ferjad Naeem , Jameel Hassan , Muzammal Naseer , Federico Tombari , Fahad Shahbaz Khan , Salman Khan

Multimodal retrieval is becoming a crucial component of modern AI applications, yet its evaluation lags behind the demands of more realistic and challenging scenarios. Existing benchmarks primarily probe surface-level semantic…

Information Retrieval · Computer Science 2025-10-01 Junjie Zhou , Ze Liu , Lei Xiong , Jin-Ge Yao , Yueze Wang , Shitao Xiao , Fenfen Lin , Miguel Hu Chen , Zhicheng Dou , Siqi Bao , Defu Lian , Yongping Xiong , Zheng Liu

As multimodal large language models (MLLMs) frequently exhibit errors in complex video reasoning scenarios, correcting these errors is critical for uncovering their weaknesses and improving performance. However, existing benchmarks lack…

Computer Vision and Pattern Recognition · Computer Science 2025-12-05 Xusen Hei , Jiali Chen , Jinyu Yang , Mengchen Zhao , Yi Cai

Generative diffusion models are developing rapidly and attracting increasing attention due to their wide range of applications. Image-to-Video (I2V) generation has become a major focus in the field of video synthesis. However, existing…

Computer Vision and Pattern Recognition · Computer Science 2025-09-30 Ailing Zhang , Lina Lei , Dehong Kong , Zhixin Wang , Jiaqi Xu , Fenglong Song , Chun-Le Guo , Chang Liu , Fan Li , Jie Chen

Despite progress in video understanding, current MLLMs struggle with counting tasks. Existing benchmarks are limited by short videos, close-set queries, lack of clue annotations, and weak multimodal coverage. In this paper, we introduce…

Computer Vision and Pattern Recognition · Computer Science 2025-07-23 Lidong Lu , Guo Chen , Zhiqi Li , Yicheng Liu , Tong Lu

Multimodal Large Language Models (MLLMs) have demonstrated significant advances in visual understanding tasks involving both images and videos. However, their capacity to comprehend human-centric video data remains underexplored, primarily…

Computer Vision and Pattern Recognition · Computer Science 2025-10-01 Yuxuan Cai , Jiangning Zhang , Zhenye Gan , Qingdong He , Xiaobin Hu , Junwei Zhu , Yabiao Wang , Chengjie Wang , Zhucun Xue , Chaoyou Fu , Xinwei He , Xiang Bai

Despite progress in video large language models (Video-LLMs), research on instructional video understanding, crucial for enhancing access to instructional content, remains insufficient. To address this, we introduce InstructionBench, an…

Computer Vision and Pattern Recognition · Computer Science 2025-07-01 Haiwan Wei , Yitian Yuan , Xiaohan Lan , Wei Ke , Lin Ma

The next frontier for video generation lies in developing models capable of zero-shot reasoning, where understanding real-world scientific laws is crucial for accurate physical outcome modeling under diverse conditions. However, existing…

Computer Vision and Pattern Recognition · Computer Science 2025-12-03 Lanxiang Hu , Abhilash Shankarampeta , Yixin Huang , Zilin Dai , Haoyang Yu , Yujie Zhao , Haoqiang Kang , Daniel Zhao , Tajana Rosing , Hao Zhang

Recent advances in video generation have enabled the synthesis of videos with strong temporal consistency and impressive visual quality, marking a crucial step toward vision foundation models. To evaluate these video generation models,…

Computer Vision and Pattern Recognition · Computer Science 2025-12-25 Xuming He , Zehao Fan , Hengjia Li , Fan Zhuo , Hankun Xu , Senlin Cheng , Di Weng , Haifeng Liu , Can Ye , Boxi Wu

Video foundation models aim to integrate video understanding, generation, editing, and instruction following within a single framework, making them a central direction for next-generation multimodal systems. However, existing evaluation…

Computer Vision and Pattern Recognition · Computer Science 2026-03-09 Jianhui Wei , Xiaotian Zhang , Yichen Li , Yuan Wang , Yan Zhang , Ziyi Chen , Zhihang Tang , Wei Xu , Zuozhu Liu

Cause-and-effect reasoning in video is a significant challenge for Vision-Language Models (VLMs), as it requires going beyond surface-level perception to a deeper understanding of causal mechanisms. However, existing benchmarks rarely…

Computer Vision and Pattern Recognition · Computer Science 2026-05-25 Mingfang Zhang , Jingjing Pan , Ashutosh Kumar , Rajat Saini , Mustafa Erdogan , Hsuan-Kung Yang , Caixin Kang , Yifei Huang , Yoichi Sato , Quan Kong

Multimodal reasoning, which integrates language and visual cues into problem solving and decision making, is a fundamental aspect of human intelligence and a crucial step toward artificial general intelligence. However, the evaluation of…

Spatio-temporal reasoning is a core capability for Multimodal Large Language Models (MLLMs) operating in the real world. As such, evaluating it precisely has become an essential challenge. However, existing spatio-temporal reasoning…

Computer Vision and Pattern Recognition · Computer Science 2026-05-22 Jinho Park , Youbin Kim , Hogun Park , Eunbyung Park

A core capability towards general embodied intelligence lies in localizing task-relevant objects from an egocentric perspective, formulated as Spatio-Temporal Video Grounding (STVG). Despite recent progress, existing STVG studies remain…

Computer Vision and Pattern Recognition · Computer Science 2026-04-07 Qi'ao Xu , Tianwen Qian , Yuqian Fu , Kailing Li , Yang Jiao , Jiacheng Zhang , Xiaoling Wang , Liang He

Humans excel at spatial-temporal reasoning, effortlessly interpreting dynamic visual events from an egocentric viewpoint. However, whether multimodal large language models (MLLMs) can similarly understand the 4D world remains uncertain.…

Computer Vision and Pattern Recognition · Computer Science 2025-04-24 Peiran Wu , Yunze Liu , Miao Liu , Junxiao Shen

The emergence of multimodal large language models (MLLMs) has triggered extensive research in model evaluation. While existing evaluation studies primarily focus on unimodal (vision-only) comprehension and reasoning capabilities, they…

Multimedia · Computer Science 2025-04-24 Xiaocui Yang , Wenfang Wu , Shi Feng , Ming Wang , Daling Wang , Yang Li , Qi Sun , Yifei Zhang , Xiaoming Fu , Soujanya Poria

While recent video world models can generate highly realistic videos, their ability to perform semantic reasoning and planning remains unclear and unquantified. We introduce Target-Bench, the first benchmark that enables comprehensive…

Video foundation models generate visually realistic and temporally coherent content, but their reliability as world simulators depends on whether they capture physical, logical, and spatial constraints. Existing metrics such as Frechet…

Computation and Language · Computer Science 2025-12-18 Zefan Cai , Haoyi Qiu , Tianyi Ma , Haozhe Zhao , Gengze Zhou , Kung-Hsiang Huang , Parisa Kordjamshidi , Minjia Zhang , Wen Xiao , Jiuxiang Gu , Nanyun Peng , Junjie Hu

Spatial understanding over continuous visual input is crucial for MLLMs to evolve into general-purpose assistants in physical environments. Yet there is still no comprehensive benchmark that holistically assesses the progress toward this…

Computer Vision and Pattern Recognition · Computer Science 2025-12-12 Jingli Lin , Runsen Xu , Shaohao Zhu , Sihan Yang , Peizhou Cao , Yunlong Ran , Miao Hu , Chenming Zhu , Yiman Xie , Yilin Long , Wenbo Hu , Dahua Lin , Tai Wang , Jiangmiao Pang