English
Related papers

Related papers: Video-R2: Reinforcing Consistent and Grounded Reas…

200 papers

Contrastive learning has been widely used to train transformer-based vision-language models for video-text alignment and multi-modal representation learning. This paper presents a new algorithm called Token-Aware Cascade contrastive…

Computer Vision and Pattern Recognition · Computer Science 2021-08-24 Jianwei Yang , Yonatan Bisk , Jianfeng Gao

Time-series reasoning remains a significant challenge in multimodal large language models (MLLMs) due to the dynamic temporal patterns, ambiguous semantics, and lack of temporal priors. In this work, we introduce TimeMaster, a reinforcement…

Machine Learning · Computer Science 2025-06-17 Junru Zhang , Lang Feng , Xu Guo , Yuhan Wu , Yabo Dong , Duanqing Xu

Large Language Models (LLMs) have shown strong capabilities through two complementary paradigms: Retrieval-Augmented Generation (RAG) for knowledge grounding and Reinforcement Learning from Verifiable Rewards (RLVR) for complex reasoning.…

Computation and Language · Computer Science 2026-04-27 Weitao Li , Boran Xiang , Xiaolong Wang , Zhinan Gou , Weizhi Ma , Yang Liu

Enhancing the multimodal reasoning capabilities of Multimodal Large Language Models (MLLMs) is a challenging task that has attracted increasing attention in the community. Recently, several studies have applied Reinforcement Learning with…

Machine Learning · Computer Science 2026-03-04 Tong Xiao , Xin Xu , Zhenya Huang , Hongyu Gao , Quan Liu , Qi Liu , Enhong Chen

Recent progress in reasoning capabilities of Multimodal Large Language Models(MLLMs) has highlighted their potential for performing complex video understanding tasks. However, in the domain of Video Anomaly Detection and Understanding…

Computer Vision and Pattern Recognition · Computer Science 2026-01-16 Chao Huang , Benfeng Wang , Wei Wang , Jie Wen , Li Shen , Wenqi Ren , Yong Xu , Xiaochun Cao

Video Models have achieved remarkable success in high-fidelity video generation with coherent motion dynamics. Analogous to the development from text generation to text-based reasoning in language modeling, the development of video models…

Computer Vision and Pattern Recognition · Computer Science 2025-11-25 Cheng Yang , Haiyuan Wan , Yiran Peng , Xin Cheng , Zhaoyang Yu , Jiayi Zhang , Junchi Yu , Xinlei Yu , Xiawu Zheng , Dongzhan Zhou , Chenglin Wu

Referring Video Object Segmentation (R-VOS) is a challenging task that aims to segment an object in a video based on a linguistic expression. Most existing R-VOS methods have a critical assumption: the object referred to must appear in the…

Computer Vision and Pattern Recognition · Computer Science 2023-08-22 Xiang Li , Jinglu Wang , Xiaohao Xu , Xiao Li , Bhiksha Raj , Yan Lu

Reasoning-oriented language models typically expose explicit reasoning as a long, front-loaded chain of "thinking" tokens before the main output, either always enabled or externally toggled at inference time. Although this can help on…

Machine Learning · Computer Science 2026-05-05 Susmit Das

Recently DeepSeek R1 has shown that reinforcement learning (RL) can substantially improve the reasoning capabilities of Large Language Models (LLMs) through a simple yet effective design. The core of R1 lies in its rule-based reward…

Computer Vision and Pattern Recognition · Computer Science 2025-04-15 Haozhan Shen , Peng Liu , Jingcheng Li , Chunxin Fang , Yibo Ma , Jiajia Liao , Qiaoli Shen , Zilun Zhang , Kangjia Zhao , Qianqian Zhang , Ruochen Xu , Tiancheng Zhao

Recent breakthroughs in reasoning language models have significantly advanced text-based reasoning. On the other hand, Multi-modal Large Language Models (MLLMs) still lag behind, hindered by their outdated internal LLMs. Upgrading these…

Computer Vision and Pattern Recognition · Computer Science 2026-03-24 Yunhao Gou , Kai Chen , Zhili Liu , Lanqing Hong , Xin Jin , Zhenguo Li , James T. Kwok , Yu Zhang

Training robust and generalizable reward models for human visual preferences is essential for aligning text-to-image and text-to-video generative models with human intent. However, current reward models often fail to generalize, and…

Computer Vision and Pattern Recognition · Computer Science 2026-04-13 Alexander Gambashidze , Li Pengyi , Matvey Skripkin , Andrey Galichin , Anton Gusarov , Konstantin Sobolev , Andrey Kuznetsov , Ivan Oseledets

Multimodal reasoning over long-horizon video is challenging due to the need for precise spatiotemporal fusion and alignment across modalities. While recent methods such as Group Relative Policy Optimization (GRPO) have shown promise in this…

Computer Vision and Pattern Recognition · Computer Science 2026-03-31 Yogesh Kulkarni , Pooyan Fazli

Multimodal Large Language Models (MLLMs) frequently hallucinate due to their reliance on fragile, linear reasoning and weak visual grounding. We propose Visual Attention Reasoning (VAR), a reinforcement learning framework that reformulates…

Artificial Intelligence · Computer Science 2026-01-27 Wei Cai , Jian Zhao , Yuchen Yuan , Tianle Zhang , Ming Zhu , Haichuan Tang , Xuelong Li

Inspired by the remarkable reasoning capabilities of Deepseek-R1 in complex textual tasks, many works attempt to incentivize similar capabilities in Multimodal Large Language Models (MLLMs) by directly applying reinforcement learning (RL).…

Machine Learning · Computer Science 2026-01-29 Shuang Chen , Yue Guo , Zhaochen Su , Yafu Li , Yulun Wu , Jiacheng Chen , Jiayu Chen , Weijie Wang , Xiaoye Qu , Yu Cheng

Developing 3D vision-language models with robust clinical reasoning remains a challenge due to the inherent complexity of volumetric medical imaging, the tendency of models to overfit superficial report patterns, and the lack of…

Computer Vision and Pattern Recognition · Computer Science 2026-02-03 Haoran Lai , Zihang Jiang , Kun Zhang , Qingsong Yao , Rongsheng Wang , Zhiyang He , Xiaodong Tao , Wei Wei , Shaohua Kevin Zhou

Multimodal reasoning is a challenging task that requires models to reason across multiple modalities to answer questions. Existing approaches have made progress by incorporating language and visual modalities into a two-stage reasoning…

Artificial Intelligence · Computer Science 2024-07-04 Cheng Tan , Jingxuan Wei , Zhangyang Gao , Linzhuang Sun , Siyuan Li , Ruifeng Guo , Bihui Yu , Stan Z. Li

Most of these text-to-video (T2V) generative models often produce single-scene video clips that depict an entity performing a particular action (e.g., 'a red panda climbing a tree'). However, it is pertinent to generate multi-scene videos…

Computer Vision and Pattern Recognition · Computer Science 2024-11-11 Hritik Bansal , Yonatan Bitton , Michal Yarom , Idan Szpektor , Aditya Grover , Kai-Wei Chang

When video reasoning requires external knowledge, many systems with large multimodal models (LMMs) adopt retrieval augmentation to supply the missing context. Appending textual or multi-clip evidence, however, forces heterogeneous signals…

Computer Vision and Pattern Recognition · Computer Science 2026-04-07 Songyuan Yang , Weijiang Yu , Ziyu Liu , Guijian Tang , Wenjing Yang , Huibin Tan , Nong Xiao

In real-world video question answering scenarios, videos often provide only localized visual cues, while verifiable answers are distributed across the open web; models therefore need to jointly perform cross-frame clue extraction, iterative…

Computer Vision and Pattern Recognition · Computer Science 2026-05-19 Chengwen Liu , Xiaomin Yu , Zhuoyue Chang , Zhe Huang , Shuo Zhang , Heng Lian , Jisheng Dang , Rui Xu , Sen Hu , Jianheng Hou , Chengwei Qin , Xiaobin Hu , Kunyi Wang , Zhi Yang , Hao Peng , Hong Peng , Ronghao Chen , Huacan Wang

Humans excel at spatial-temporal reasoning, effortlessly interpreting dynamic visual events from an egocentric viewpoint. However, whether multimodal large language models (MLLMs) can similarly understand the 4D world remains uncertain.…

Computer Vision and Pattern Recognition · Computer Science 2025-04-24 Peiran Wu , Yunze Liu , Miao Liu , Junxiao Shen