English
Related papers

Related papers: Reinforcement Learning Tuning for VideoLLMs: Rewar…

200 papers

Reinforcement Learning (RL) benefits Large Language Models (LLMs) for complex reasoning. Inspired by this, we explore integrating spatio-temporal specific rewards into Multimodal Large Language Models (MLLMs) to address the unique…

Computer Vision and Pattern Recognition · Computer Science 2025-11-12 Xinhao Li , Ziang Yan , Desen Meng , Lu Dong , Xiangyu Zeng , Yinan He , Yali Wang , Yu Qiao , Yi Wang , Limin Wang

Reinforcement fine-tuning (RFT) has shown great promise in achieving humanlevel reasoning capabilities of Large Language Models (LLMs), and has recently been extended to MLLMs. Nevertheless, reasoning about videos, which is a fundamental…

Computer Vision and Pattern Recognition · Computer Science 2025-10-15 Qi Wang , Yanrui Yu , Ye Yuan , Rui Mao , Tianfei Zhou

The growing disparity between the exponential scaling of computational resources and the finite growth of high-quality text data now constrains conventional scaling approaches for large language models (LLMs). To address this challenge, we…

Despite advances in reinforcement learning (RL)-based video reasoning with large language models (LLMs), data collection and fine-tuning remain significant challenges. These methods often rely on large-scale supervised fine-tuning (SFT)…

Computer Vision and Pattern Recognition · Computer Science 2025-10-27 Ziyang Wang , Jaehong Yoon , Shoubin Yu , Md Mohaiminul Islam , Gedas Bertasius , Mohit Bansal

Recent advancements in Chain of Thought (COT) generation have significantly improved the reasoning capabilities of Large Language Models (LLMs), with reinforcement learning (RL) emerging as an effective post-training approach. Multimodal…

Computer Vision and Pattern Recognition · Computer Science 2025-04-01 Yi Chen , Yuying Ge , Rui Wang , Yixiao Ge , Lu Qiu , Ying Shan , Xihui Liu

The application of reinforcement learning (RL) to enhance the reasoning capabilities of Multimodal Large Language Models (MLLMs) constitutes a rapidly advancing research area. While MLLMs extend Large Language Models (LLMs) to handle…

Artificial Intelligence · Computer Science 2025-05-22 Guanghao Zhou , Panjia Qiu , Cen Chen , Jie Wang , Zheming Yang , Jian Xu , Minghui Qiu

This paper investigates Reinforcement Learning (RL) approaches to enhance the reasoning capabilities of Large Language Model (LLM) agents in long-horizon, multi-turn scenarios. Although RL algorithms such as Group Relative Policy…

Enhancing the temporal understanding of Multimodal Large Language Models (MLLMs) is essential for advancing long-form video analysis, enabling tasks such as temporal localization, action detection, and time-sensitive question answering.…

Computer Vision and Pattern Recognition · Computer Science 2026-04-15 Tao Wu , Li Yang , Gen Zhan , Yabin Zhang , Yiting Liao , Junlin Li , Deliang Fu , Li Zhang , Limin Wang

Reinforcement learning (RL) has emerged as a promising approach for eliciting reasoning chains before generating final answers. However, multimodal large language models (MLLMs) generate reasoning that lacks integration of visual…

Computer Vision and Pattern Recognition · Computer Science 2026-01-05 Omar Sharif , Eftekhar Hossain , Patrick Ng

The video reasoning ability of multimodal large language models (MLLMs) is crucial for downstream tasks like video question answering and temporal grounding. While recent approaches have explored text-based chain-of-thought (CoT) reasoning…

Computer Vision and Pattern Recognition · Computer Science 2025-09-04 Haoji Zhang , Xin Gu , Jiawen Li , Chixiang Ma , Sule Bai , Chubin Zhang , Bowen Zhang , Zhichao Zhou , Dongliang He , Yansong Tang

Inspired by DeepSeek-R1's success in eliciting reasoning abilities through rule-based reinforcement learning (RL), we introduce Video-R1 as the first attempt to systematically explore the R1 paradigm for incentivizing video reasoning within…

Computer Vision and Pattern Recognition · Computer Science 2025-10-23 Kaituo Feng , Kaixiong Gong , Bohao Li , Zonghao Guo , Yibing Wang , Tianshuo Peng , Junfei Wu , Xiaoying Zhang , Benyou Wang , Xiangyu Yue

Reinforcement Learning (RL) has emerged as a transformative approach for aligning and enhancing Large Language Models (LLMs), addressing critical challenges in instruction following, ethical alignment, and reasoning capabilities. This…

Artificial Intelligence · Computer Science 2025-07-08 Saksham Sahai Srivastava , Vaneet Aggarwal

Visual reasoning abilities play a crucial role in understanding complex multimodal data, advancing both domain-specific applications and artificial general intelligence (AGI). Existing methods enhance Vision-Language Models (VLMs) through…

Computer Vision and Pattern Recognition · Computer Science 2025-10-07 Huajie Tan , Yuheng Ji , Xiaoshuai Hao , Xiansheng Chen , Pengwei Wang , Zhongyuan Wang , Shanghang Zhang

Large vision-language models (VLMs) fine-tuned on specialized visual instruction-following data have exhibited impressive language reasoning capabilities across various scenarios. However, this fine-tuning paradigm may not be able to…

Artificial Intelligence · Computer Science 2024-10-10 Yuexiang Zhai , Hao Bai , Zipeng Lin , Jiayi Pan , Shengbang Tong , Yifei Zhou , Alane Suhr , Saining Xie , Yann LeCun , Yi Ma , Sergey Levine

Enhancing the multimodal reasoning capabilities of Multimodal Large Language Models (MLLMs) is a challenging task that has attracted increasing attention in the community. Recently, several studies have applied Reinforcement Learning with…

Machine Learning · Computer Science 2026-03-04 Tong Xiao , Xin Xu , Zhenya Huang , Hongyu Gao , Quan Liu , Qi Liu , Enhong Chen

Looped Language Models (LoopLMs) perform multi-step latent reasoning prior to token generation and outperform conventional LLMs on reasoning benchmarks at smaller parameter budgets. However, attempts to further improve LoopLM reasoning with…

Machine Learning · Computer Science 2026-05-29 Jonathan Williams , Esin Tureci

Recently DeepSeek R1 has shown that reinforcement learning (RL) can substantially improve the reasoning capabilities of Large Language Models (LLMs) through a simple yet effective design. The core of R1 lies in its rule-based reward…

Computer Vision and Pattern Recognition · Computer Science 2025-04-15 Haozhan Shen , Peng Liu , Jingcheng Li , Chunxin Fang , Yibo Ma , Jiajia Liao , Qiaoli Shen , Zilun Zhang , Kangjia Zhao , Qianqian Zhang , Ruochen Xu , Tiancheng Zhao

Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as a powerful paradigm for post-training large language models (LLMs), achieving state-of-the-art performance on tasks with structured, verifiable answers. Applying…

Computer Vision and Pattern Recognition · Computer Science 2025-06-06 Yiqing Liang , Jielin Qiu , Wenhao Ding , Zuxin Liu , James Tompkin , Mengdi Xu , Mengzhou Xia , Zhengzhong Tu , Laixi Shi , Jiacheng Zhu

Large language models (LLMs) trained via pretraining and supervised fine-tuning (SFT) can still produce harmful and misaligned outputs, or struggle in domains like math and coding. Reinforcement learning (RL)-based post-training methods,…

Computation and Language · Computer Science 2026-05-19 Zhichao Wang , Kiran Ramnath , Bin Bi , Shiva Kumar Pentyala , Sougata Chaudhuri , Shubham Mehrotra , Zixu , Zhu , Xiang-Bo Mao , Sitaram Asur , Na , Cheng

This paper introduces TempSamp-R1, a new reinforcement fine-tuning framework designed to improve the effectiveness of adapting multimodal large language models (MLLMs) to video temporal grounding tasks. We reveal that existing reinforcement…

Computer Vision and Pattern Recognition · Computer Science 2025-09-26 Yunheng Li , Jing Cheng , Shaoyong Jia , Hangyi Kuang , Shaohui Jiao , Qibin Hou , Ming-Ming Cheng
‹ Prev 1 2 3 10 Next ›