English
Related papers

Related papers: The Multi-Modal Video Reasoning and Analyzing Comp…

200 papers

We propose Skywork-VL Reward, a multimodal reward model that provides reward signals for both multimodal understanding and reasoning tasks. Our technical approach comprises two key components: First, we construct a large-scale multimodal…

Computer Vision and Pattern Recognition · Computer Science 2025-06-10 Xiaokun Wang , Peiyu Wang , Jiangbo Pei , Wei Shen , Yi Peng , Yunzhuo Hao , Weijie Qiu , Ai Jian , Tianyidan Xie , Xuchen Song , Yang Liu , Yahui Zhou

Multimodal learning holds promise for richer information extraction by capturing dependencies across data sources. Yet, current training methods often underperform due to modality competition, a phenomenon where modalities contend for…

Enhancing the multimodal reasoning capabilities of Multimodal Large Language Models (MLLMs) is a challenging task that has attracted increasing attention in the community. Recently, several studies have applied Reinforcement Learning with…

Machine Learning · Computer Science 2026-03-04 Tong Xiao , Xin Xu , Zhenya Huang , Hongyu Gao , Quan Liu , Qi Liu , Enhong Chen

This paper presents an after-the-fact summary of the participation of the vitrivr system to the 2018 Video Browser Showdown. A particular focus is on additions made since the original publication and the systems performance during the…

Multimedia · Computer Science 2018-05-08 Luca Rossetto , Ivan Giangreco , Ralph Gasser , Heiko Schuldt

Multimodal reward models (MRMs) play a crucial role in the training, inference, and evaluation of Large Vision Language Models (LVLMs) by assessing response quality. However, existing benchmarks for evaluating MRMs in the video domain…

Computer Vision and Pattern Recognition · Computer Science 2025-09-03 Zhihong Zhang , Xiaojian Huang , Jin Xu , Zhuodong Luo , Xinzhi Wang , Jiansheng Wei , Xuejin Chen

Multi-object tracking (MOT) has traditionally focused on estimating trajectories of all objects in a video, without selectively reasoning about user-specified targets under semantic instructions. In this work, we introduce a query-driven…

Computer Vision and Pattern Recognition · Computer Science 2026-03-17 Tajamul Ashraf , Tavaheed Tariq , Sonia Yadav , Abrar Ul Riyaz , Wasif Tak , Moloud Abdar , Janibul Bashir

This companion paper reports the ICPR 2026 TVRID competition on privacy-aware top-view person re-identification. We present the competition setting, the released RGB-Depth dataset, and a summary of final results with descriptions of the top…

Computer Vision and Pattern Recognition · Computer Science 2026-05-07 Raphaël Delécluse , Hazem Wannous , Laurent Guimas

In this report, we present the winning solution that achieved the 1st place in the Complex Video Reasoning & Robustness Evaluation Challenge 2025. This challenge evaluates the ability to generate accurate natural language answers to…

Computer Vision and Pattern Recognition · Computer Science 2025-06-30 Umihiro Kamoto , Tatsuya Ishibashi , Noriyuki Kugo

Efficient long-video understanding~(LVU) remains a challenging task in computer vision. Current long-context vision-language models~(LVLMs) suffer from information loss due to compression and brute-force downsampling. While…

Computer Vision and Pattern Recognition · Computer Science 2025-06-23 Huaying Yuan , Zheng Liu , Minghao Qin , Hongjin Qian , Yan Shu , Zhicheng Dou , Ji-Rong Wen , Nicu Sebe

Multi-modal object tracking (MMOT) is an emerging field that combines data from various modalities, \eg vision (RGB), depth, thermal infrared, event, language and audio, to estimate the state of an arbitrary object in a video sequence. It…

Computer Vision and Pattern Recognition · Computer Science 2024-06-03 Chunhui Zhang , Li Liu , Hao Wen , Xi Zhou , Yanfeng Wang

Recently, significant progress has been made in multi-modal continual learning, aiming to learn new tasks sequentially in multi-modal settings while preserving performance on previously learned ones. However, existing methods mainly focus…

Multimedia · Computer Science 2026-03-10 Yuyang Hong , Qi Yang , Tao Zhang , Zili Wang , Zhaojin Fu , Kun Ding , Bin Fan , Shiming Xiang

Multi-modal Large Language Models (MLLMs) are increasingly prominent in the field of artificial intelligence. These models not only excel in traditional vision-language tasks but also demonstrate impressive performance in contemporary…

Computer Vision and Pattern Recognition · Computer Science 2023-12-06 Xiaotian Han , Quanzeng You , Yongfei Liu , Wentao Chen , Huangjie Zheng , Khalil Mrini , Xudong Lin , Yiqi Wang , Bohan Zhai , Jianbo Yuan , Heng Wang , Hongxia Yang

This paper describes the MediaEval 2021 Predicting Media Memorability}task, which is in its 4th edition this year, as the prediction of short-term and long-term video memorability remains a challenging task. In 2021, two datasets of videos…

Multimodal reasoning remains a fundamental challenge in artificial intelligence. Despite substantial advances in text-based reasoning, even state-of-the-art models such as GPT-o3 struggle to maintain strong performance in multimodal…

Computation and Language · Computer Science 2025-09-09 Hao Liang , Ruitao Wu , Bohan Zeng , Junbo Niu , Wentao Zhang , Bin Dong

The rapid development of diffusion models has greatly advanced AI-generated videos in terms of length and consistency recently, yet assessing AI-generated videos still remains challenging. Previous approaches have often focused on…

Computer Vision and Pattern Recognition · Computer Science 2025-01-07 Jiaze Li , Haoran Xu , Shiding Zhu , Junwei He , Haozhao Wang

Visual Question Answering (VQA) has emerged as one of the most challenging tasks in artificial intelligence due to its multi-modal nature. However, most existing VQA methods are incapable of handling Knowledge-based Visual Question…

Computer Vision and Pattern Recognition · Computer Science 2023-12-21 Chengxiang Yin , Zhengping Che , Kun Wu , Zhiyuan Xu , Jian Tang

Video causal reasoning aims to achieve a high-level understanding of video content from a causal perspective. However, current video reasoning tasks are limited in scope, primarily executed in a question-answering paradigm and focusing on…

Computer Vision and Pattern Recognition · Computer Science 2024-12-30 Tieyuan Chen , Huabin Liu , Tianyao He , Yihang Chen , Chaofan Gan , Xiao Ma , Cheng Zhong , Yang Zhang , Yingxue Wang , Hui Lin , Weiyao Lin

Multimodal learning aims to imitate human beings to acquire complementary information from multiple modalities for various downstream tasks. However, traditional aggregation-based multimodal fusion methods ignore the inter-modality…

Computer Vision and Pattern Recognition · Computer Science 2023-05-17 Heqing Zou , Meng Shen , Chen Chen , Yuchen Hu , Deepu Rajan , Eng Siong Chng

Analyzing video for traffic categorization is an important pillar of Intelligent Transport Systems. However, it is difficult to analyze and predict traffic based on image frames because the representation of each frame may vary…

Computer Vision and Pattern Recognition · Computer Science 2019-02-19 Somdip Dey , Amit K. Singh , Dilip K. Prasad , Klaus D. McDonald-Maier