English
Related papers

Related papers: Watching, Reasoning, and Searching: A Video Deep R…

200 papers

Multi-modal language models (LM) have recently shown promising performance in high-level reasoning tasks on videos. However, existing methods still fall short in tasks like causal or compositional spatiotemporal reasoning over actions, in…

Computer Vision and Pattern Recognition · Computer Science 2024-01-23 Apratim Bhattacharyya , Sunny Panchal , Mingu Lee , Reza Pourreza , Pulkit Madan , Roland Memisevic

Mathematical reasoning in real-world video settings presents a fundamentally different challenge than in static images or text. It requires interpreting fine-grained visual information, accurately reading handwritten or digital text, and…

Computer Vision and Pattern Recognition · Computer Science 2025-06-25 Hanoona Rasheed , Abdelrahman Shaker , Anqi Tang , Muhammad Maaz , Ming-Hsuan Yang , Salman Khan , Fahad Shahbaz Khan

Understanding long-form video content presents significant challenges due to its temporal complexity and the substantial computational resources required. In this work, we propose an agent-based approach to enhance both the efficiency and…

Computer Vision and Pattern Recognition · Computer Science 2024-10-29 Sullam Jeoung , Goeric Huybrechts , Bhavana Ganesh , Aram Galstyan , Sravan Bodapati

With the increasing prevalence of video content, effectively understanding and answering questions about long form videos has become essential for numerous applications. Although large vision language models (LVLMs) have enhanced…

Computer Vision and Pattern Recognition · Computer Science 2026-04-13 Urjitkumar Patel , Fang-Chun Yeh , Chinmay Gondhalekar

Recent advances in video generation have enabled the synthesis of videos with strong temporal consistency and impressive visual quality, marking a crucial step toward vision foundation models. To evaluate these video generation models,…

Computer Vision and Pattern Recognition · Computer Science 2025-12-25 Xuming He , Zehao Fan , Hengjia Li , Fan Zhuo , Hankun Xu , Senlin Cheng , Di Weng , Haifeng Liu , Can Ye , Boxi Wu

Video Question Answering (VideoQA) has made significant strides by leveraging multimodal learning to align visual and textual modalities. However, current benchmarks overwhelmingly focus on questions answerable through explicit visual…

Computer Vision and Pattern Recognition · Computer Science 2026-03-31 Sirnam Swetha , Rohit Gupta , Parth Parag Kulkarni , David G Shatwell , Jeffrey A Chan Santiago , Nyle Siddiqui , Joseph Fioresi , Mubarak Shah

We introduce ReXTime, a benchmark designed to rigorously test AI models' ability to perform temporal reasoning within video events. Specifically, ReXTime focuses on reasoning across time, i.e. human-like understanding when the question and…

Computer Vision and Pattern Recognition · Computer Science 2024-07-03 Jr-Jen Chen , Yu-Chien Liao , Hsi-Che Lin , Yu-Chu Yu , Yen-Chun Chen , Yu-Chiang Frank Wang

Web agents such as Deep Research have demonstrated superhuman cognitive abilities, capable of solving highly challenging information-seeking problems. However, most research remains primarily text-centric, overlooking visual information in…

Information Retrieval · Computer Science 2025-09-03 Xinyu Geng , Peng Xia , Zhen Zhang , Xinyu Wang , Qiuchen Wang , Ruixue Ding , Chenxi Wang , Jialong Wu , Yida Zhao , Kuan Li , Yong Jiang , Pengjun Xie , Fei Huang , Jingren Zhou

Visual reasoning, the capability to interpret visual input in response to implicit text query through multi-step reasoning, remains a challenge for deep learning models due to the lack of relevant benchmarks. Previous work in visual…

Computer Vision and Pattern Recognition · Computer Science 2025-05-20 Yiqing Shen , Chenjia Li , Chenxiao Fan , Mathias Unberath

Even in the era of rapid advances in large models, video understanding remains a highly challenging task. Compared to texts or images, videos commonly contain more information with redundancy, requiring large models to properly allocate…

Computer Vision and Pattern Recognition · Computer Science 2025-04-29 Shiwen Cao , Zhaoxing Zhang , Junming Jiao , Juyi Qiao , Guowen Song , Rong Shen , Xiangbing Meng

Large Language Models (LLMs) and Vision Language Models (VLMs) possess extensive knowledge and exhibit promising reasoning abilities, however, they still struggle to perform well in complex, dynamic environments. Real-world tasks require…

In long-video understanding, conventional uniform frame sampling often fails to capture key visual evidence, leading to degraded performance and increased hallucinations. To address this, recent agentic thinking-with-videos paradigms have…

Computer Vision and Pattern Recognition · Computer Science 2026-05-25 Wenqi Liu , Yunxiao Wang , Shijie Ma , Meng Liu , Qile Su , Tianke Zhang , Haonan Fan , Changyi Liu , Kaiyu Jiang , Jiankang Chen , Kaiyu Tang , Bin Wen , Fan Yang , Tingting Gao , Han Li , Yinwei Wei , Xuemeng Song

Long video understanding (LVU) is challenging because answering real-world queries often depends on sparse, temporally dispersed cues buried in hours of mostly redundant and irrelevant content. While agentic pipelines improve video…

Computer Vision and Pattern Recognition · Computer Science 2025-12-08 Ziyang Wang , Honglu Zhou , Shijie Wang , Junnan Li , Caiming Xiong , Silvio Savarese , Mohit Bansal , Michael S. Ryoo , Juan Carlos Niebles

Multimodal large language models (MLLMs) have substantially advanced video misinformation detection through unified multimodal reasoning, but they often rely on fixed-depth inference and place excessive trust in internally generated…

Artificial Intelligence · Computer Science 2026-02-27 Zehao Li , Hongwei Yu , Hao Jiang , Qiang Sheng , Yilong Xu , Baolong Bi , Yang Li , Zhenlong Yuan , Yujun Cai , Zhaoqi Wang

Information retrieval is a cornerstone of modern knowledge acquisition, enabling billions of queries each day across diverse domains. However, traditional keyword-based search engines are increasingly inadequate for handling complex,…

Large multimodal models (LMMs) are processing increasingly longer and richer inputs. Albeit the progress, few public benchmark is available to measure such development. To mitigate this gap, we introduce LongVideoBench, a question-answering…

Computer Vision and Pattern Recognition · Computer Science 2024-07-23 Haoning Wu , Dongxu Li , Bei Chen , Junnan Li

Reasoning over dynamic visual content remains a central challenge for multimodal large language models. Recent thinking models generate explicit reasoning traces for interpretability; however, their reasoning often appears convincing while…

Computer Vision and Pattern Recognition · Computer Science 2025-12-09 Muhammad Maaz , Hanoona Rasheed , Fahad Shahbaz Khan , Salman Khan

As multimodal content continues to expand at a rapid pace, audio retrieval has emerged as a key enabling technology for media search, content organization, and intelligent assistants. However, most existing benchmarks concentrate on…

Artificial Intelligence · Computer Science 2026-05-07 Honglei Zhang , Yuting Chen , Chenpeng Hu , Siyue Zhang , Yilei Shi

While Reinforcement Learning with Verifiable Reward (RLVR) significantly advances image reasoning in Large Vision-Language Models (LVLMs), its application to complex video reasoning remains underdeveloped. This gap stems primarily from a…

Computer Vision and Pattern Recognition · Computer Science 2025-10-02 Congzhi Zhang , Zhibin Wang , Yinchao Ma , Jiawei Peng , Yihan Wang , Qiang Zhou , Jun Song , Bo Zheng

This paper introduces MovieCORE, a novel video question answering (VQA) dataset designed to probe deeper cognitive understanding of movie content. Unlike existing datasets that focus on surface-level comprehension, MovieCORE emphasizes…

Computation and Language · Computer Science 2025-09-19 Gueter Josmy Faure , Min-Hung Chen , Jia-Fong Yeh , Ying Cheng , Hung-Ting Su , Yung-Hao Tang , Shang-Hong Lai , Winston H. Hsu