English

A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding

Computer Vision and Pattern Recognition 2026-03-17 v1

Abstract

Multimodal Large Language Models have achieved strong performance in single-video understanding, yet their ability to reason across multiple videos remains limited. Existing approaches typically concatenate multiple videos into a single input and perform direct inference, which introduces training-inference mismatch, information loss from frame compression, and a lack of explicit cross-video coordination. Meanwhile, current multi-video benchmarks primarily emphasize event-level comparison, leaving identity-level matching, fine-grained discrimination, and structured multi-step reasoning underexplored. To address these gaps, we introduce MVX-Bench, a Multi-Video Cross-Dimension Benchmark that reformulates 11 classical computer vision tasks into a unified multi-video question-answering framework, comprising 1,442 questions over 4,255 videos from diverse real-world datasets. We further propose SAMA, a Skill-Augmented Agentic Framework for Multi-Video Understanding, which integrates visual tools, task-specific skills, and a conflict-aware verification mechanism to enable iterative and structured reasoning. Experimental results show that SAMA outperforms strong open-source baselines and GPT on MVX-Bench, and ablations validate the effectiveness of skill design and conflict resolution.

Keywords

Cite

@article{arxiv.2603.14733,
  title  = {A Skill-augmented Agentic Framework and Benchmark for Multi-Video Understanding},
  author = {Yue Zhang and Liqiang Jing and Jia Li and Yapeng Tian and Xinya Du and Yunhui Guo and Vibhav Gogate},
  journal= {arXiv preprint arXiv:2603.14733},
  year   = {2026}
}
R2 v1 2026-07-01T11:21:17.586Z