English
Related papers

Related papers: Spotlight on Token Perception for Multimodal Reinf…

200 papers

Reinforcement learning from verifiable rewards (RLVR) has recently been extended from text-only LLMs to vision-language models (VLMs) to elicit long-chain multimodal reasoning. However, RLVR-trained VLMs still exhibit two persistent failure…

Computer Vision and Pattern Recognition · Computer Science 2025-12-16 Hoang Anh Just , Yifei Fan , Handong Zhao , Jiuxiang Gu , Ruiyi Zhang , Simon Jenni , Kushal Kafle , Ruoxi Jia , Jing Shi

Reinforcement Learning with Verifiable Rewards (RLVR) has been shown effective in enhancing the visual reflection and reasoning capabilities of Large Multimodal Models (LMMs). However, existing datasets are predominantly derived from either…

Machine Learning · Computer Science 2026-02-20 Haoxiang Sun , Lizhen Xu , Bing Zhao , Wotao Yin , Wei Wang , Boyu Yang , Rui Wang , Hu Wei

Reinforcement learning with verifiable rewards (RLVR) has proven effective in training large reasoning models (LRMs) by leveraging answer-verifiable signals to guide policy optimization, which, however, suffers from high annotation costs.…

Machine Learning · Computer Science 2025-12-16 Shenzhi Yang , Guangcheng Zhu , Xing Zheng , Yingfan MA , Zhongqi Chen , Bowen Song , Weiqiang Wang , Junbo Zhao , Gang Chen , Haobo Wang

While Large Language Models (LLMs) excel at reasoning on text and Vision-Language Models (VLMs) are highly effective for visual perception, applying those models for visual instruction-based planning remains a widely open problem. In this…

Machine Learning · Computer Science 2025-09-11 Mohamed Salim Aissi , Clemence Grislain , Mohamed Chetouani , Olivier Sigaud , Laure Soulier , Nicolas Thome

Reinforcement Learning with Human Feedback (RLHF) has been the dominant approach for improving the reasoning capabilities of Large Language Models (LLMs). Recently, Reinforcement Learning with Verifiable Rewards (RLVR) has simplified this…

Computation and Language · Computer Science 2025-10-10 Yining Wang , Jinman Zhao , Chuangxin Zhao , Shuhao Guan , Gerald Penn , Shinan Liu

Despite tremendous recent advances in large model reasoning ability, vision-language models (VLMs) still struggle with detailed visual reasoning, especially when compute resources are limited. To address this challenge, we draw inspiration…

Machine Learning · Computer Science 2025-08-06 Sunil Kumar , Bowen Zhao , Leo Dirac , Paulina Varshavskaya

Recent large vision-language models (LVLMs) have demonstrated impressive reasoning ability by generating long chain-of-thought (CoT) responses. However, CoT reasoning in multimodal contexts is highly vulnerable to visual hallucination…

Computer Vision and Pattern Recognition · Computer Science 2026-03-10 Yongchang Zhang , Oliver Ma , Tianyi Liu , Guangquan Zhou , Yang Chen

Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as a powerful paradigm for post-training large language models (LLMs), achieving state-of-the-art performance on tasks with structured, verifiable answers. Applying…

Computer Vision and Pattern Recognition · Computer Science 2025-06-06 Yiqing Liang , Jielin Qiu , Wenhao Ding , Zuxin Liu , James Tompkin , Mengdi Xu , Mengzhou Xia , Zhengzhong Tu , Laixi Shi , Jiacheng Zhu

Process Reward Models (PRMs) provide step-level supervision that improves the reliability of reasoning in large language models. While PRMs have been extensively studied in text-based domains, their extension to Vision Language Models…

Artificial Intelligence · Computer Science 2025-10-08 Brandon Ong , Tej Deep Pala , Vernon Toh , William Chandra Tjhi , Soujanya Poria

Reinforcement Learning has significantly advanced the reasoning capabilities of Multimodal Large Language Models (MLLMs), yet the resulting policies remain brittle against real-world visual degradations such as blur, compression artifacts,…

Computer Vision and Pattern Recognition · Computer Science 2026-05-12 Rui Liu , Dian Yu , Haolin Liu , Yucheng Shi , Tong Zheng , Runpeng Dai , Haitao Mi , Pratap Tokekar , Leoweiliang

Large Vision-Language Models (LVLMs) or multimodal large language models represent a significant advancement in artificial intelligence, enabling systems to understand and generate content across both visual and textual modalities. While…

Machine Learning · Computer Science 2025-09-09 Thanh Thi Nguyen , Campbell Wilson , Janis Dalins

Reinforcement Learning with Verifiable Rewards (RLVR) has achieved remarkable success in improving autoregressive models, especially in domains requiring correctness like mathematical reasoning and code generation. However, directly…

Machine Learning · Computer Science 2026-03-03 Chenxing Wei , Jiazhen Kang , Hong Wang , Jianqing Zhang , Hao Jiang , Xiaolong Xu , Ningyuan Sun , Ying He , F. Richard Yu , Yao Shu , Bo Jiang

Video reasoning segmentation (VRS) endeavors to delineate referred objects in videos guided by implicit instructions that encapsulate human intent and temporal logic. Previous approaches leverage large vision language models (LVLMs) to…

Computer Vision and Pattern Recognition · Computer Science 2026-03-05 Sitong Gong , Lu Zhang , Yunzhi Zhuge , Xu Jia , Pingping Zhang , Huchuan Lu

This paper investigates Reinforcement Learning (RL) approaches to enhance the reasoning capabilities of Large Language Model (LLM) agents in long-horizon, multi-turn scenarios. Although RL algorithms such as Group Relative Policy…

The diagnosis of pathological images is often limited by expert availability and regional disparities, highlighting the importance of automated diagnosis using Vision-Language Models (VLMs). Traditional multimodal models typically emphasize…

Computer Vision and Pattern Recognition · Computer Science 2025-04-21 Jianyu Wu , Hao Yang , Xinhua Zeng , Guibing He , Zhiyu Chen , Zihui Li , Xiaochuan Zhang , Yangyang Ma , Run Fang , Yang Liu

Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced the reasoning capabilities of Large Language Models (LLMs) and is now being applied to Vision-Language Models (VLMs). However, vanilla RLVR for VLMs verifies…

Computer Vision and Pattern Recognition · Computer Science 2025-11-25 Chi Zhang , Haibo Qiu , Qiming Zhang , Yufei Xu , Zhixiong Zeng , Siqi Yang , Peng Shi , Lin Ma , Jing Zhang

Chain-of-Thought (CoT) reasoning has significantly advanced the problem-solving capabilities of Large Language Models (LLMs), yet conventional CoT often exhibits internal determinism during decoding, limiting exploration of plausible…

Artificial Intelligence · Computer Science 2025-12-09 Jindi Lv , Yuhao Zhou , Zheng Zhu , Xiaofeng Wang , Guan Huang , Jiancheng Lv

Reinforcement Learning with Verifiable Rewards (RLVR) has improved the reasoning abilities of Large Language Models (LLMs) by using rule-based binary feedback. However, current RLVR methods typically assign the same reward to every token.…

Machine Learning · Computer Science 2025-10-21 Guofu Xie , Yunsheng Shi , Hongtao Tian , Ting Yao , Xiao Zhang

Existing Multimodal Large Language Models (MLLMs) suffer from increased inference costs due to the additional vision tokens introduced by image inputs. In this work, we propose Visual Consistency Learning (ViCO), a novel training algorithm…

Computer Vision and Pattern Recognition · Computer Science 2025-12-16 Long Cui , Weiyun Wang , Jie Shao , Zichen Wen , Gen Luo , Linfeng Zhang , Yanting Zhang , Yu Qiao , Wenhai Wang

Reinforcement learning with verifiable rewards (RLVR) has recently enhanced the reasoning capabilities of large language models (LLMs), particularly for mathematical problem solving. However, a fundamental limitation remains: as the…

Machine Learning · Computer Science 2025-11-03 Wenhao Deng , Long Wei , Chenglei Yu , Tailin Wu