English
Related papers

Related papers: LaViDa-R1: Advancing Reasoning for Unified Multimo…

200 papers

Recent advances in general medical AI have made significant strides, but existing models often lack the reasoning capabilities needed for complex medical decision-making. This paper presents GMAI-VL-R1, a multimodal medical reasoning model…

Computer Vision and Pattern Recognition · Computer Science 2025-04-03 Yanzhou Su , Tianbin Li , Jiyao Liu , Chenglong Ma , Junzhi Ning , Cheng Tang , Sibo Ju , Jin Ye , Pengcheng Chen , Ming Hu , Shixiang Tang , Lihao Liu , Bin Fu , Wenqi Shao , Xiaowei Hu , Xiangwen Liao , Yuanfeng Ji , Junjun He

Large Vision-Language Models (LVLMs) have recently advanced robotic manipulation by leveraging vision for scene perception and language for instruction following. However, existing methods rely heavily on costly human-annotated training…

The rapid progress of auto-regressive vision-language models (VLMs) has inspired growing interest in vision-language-action models (VLA) for robotic manipulation. Recently, masked diffusion models, a paradigm distinct from autoregressive…

Robotics · Computer Science 2025-09-11 Yuqing Wen , Hebei Li , Kefan Gu , Yucheng Zhao , Tiancai Wang , Xiaoyan Sun

Recent works on large language models (LLMs) have successfully demonstrated the emergence of reasoning capabilities via reinforcement learning (RL). Although recent efforts leverage group relative policy optimization (GRPO) for MLLMs…

Computation and Language · Computer Science 2025-06-18 Shilin Xu , Yanwei Li , Rui Yang , Tao Zhang , Yueyi Sun , Wei Chow , Linfeng Li , Hang Song , Qi Xu , Yunhai Tong , Xiangtai Li , Hao Fei

Building on the success of text-based reasoning models like DeepSeek-R1, extending these capabilities to multimodal reasoning holds great promise. While recent works have attempted to adapt DeepSeek-R1-style reinforcement learning (RL)…

Computer Vision and Pattern Recognition · Computer Science 2025-06-10 Jie Yang , Feipeng Ma , Zitian Wang , Dacheng Yin , Kang Rong , Fengyun Rao , Ruimao Zhang

Large Language Models (LLMs) demonstrate their reasoning ability through chain-of-thought (CoT) generation. However, LLM's autoregressive decoding may limit the ability to revisit and refine earlier tokens in a holistic manner, which can…

Machine Learning · Computer Science 2026-04-24 Haoqiang Kang , Yizhe Zhang , Nikki Lijing Kuang , Nicklas Majamaki , Navdeep Jaitly , Yi-An Ma , Lianhui Qin

Recently DeepSeek R1 has shown that reinforcement learning (RL) can substantially improve the reasoning capabilities of Large Language Models (LLMs) through a simple yet effective design. The core of R1 lies in its rule-based reward…

Computer Vision and Pattern Recognition · Computer Science 2025-04-15 Haozhan Shen , Peng Liu , Jingcheng Li , Chunxin Fang , Yibo Ma , Jiajia Liao , Qiaoli Shen , Zilun Zhang , Kangjia Zhao , Qianqian Zhang , Ruochen Xu , Tiancheng Zhao

In recent years, significant progress has been made in the field of surgical scene understanding, particularly in the task of Visual Question Localized-Answering in robotic surgery (Surgical-VQLA). However, existing Surgical-VQLA models…

Computer Vision and Pattern Recognition · Computer Science 2025-06-25 Pengfei Hao , Shuaibo Li , Hongqiu Wang , Zhizhuo Kou , Junhang Zhang , Guang Yang , Lei Zhu

Inspired by the remarkable reasoning capabilities of Deepseek-R1 in complex textual tasks, many works attempt to incentivize similar capabilities in Multimodal Large Language Models (MLLMs) by directly applying reinforcement learning (RL).…

Machine Learning · Computer Science 2026-01-29 Shuang Chen , Yue Guo , Zhaochen Su , Yafu Li , Yulun Wu , Jiacheng Chen , Jiayu Chen , Weijie Wang , Xiaoye Qu , Yu Cheng

We introduce Skywork R1V, a multimodal reasoning model extending the an R1-series Large language models (LLM) to visual modalities via an efficient multimodal transfer method. Leveraging a lightweight visual projector, Skywork R1V…

Computer Vision and Pattern Recognition · Computer Science 2025-06-10 Yi Peng , Peiyu Wang , Xiaokun Wang , Yichen Wei , Jiangbo Pei , Weijie Qiu , Ai Jian , Yunzhuo Hao , Jiachun Pan , Tianyidan Xie , Li Ge , Rongxian Zhuang , Xuchen Song , Yang Liu , Yahui Zhou

We propose DiFFPO, Diffusion Fast and Furious Policy Optimization, a unified framework for training masked diffusion large language models (dLLMs) to reason not only better (furious), but also faster via reinforcement learning (RL). We…

Machine Learning · Computer Science 2026-01-13 Hanyang Zhao , Dawen Liang , Wenpin Tang , David Yao , Nathan Kallus

The paradigm of Large Language Models (LLMs) is currently defined by auto-regressive (AR) architectures, which generate text through a sequential ``brick-by-brick'' process. Despite their success, AR models are inherently constrained by a…

Improving the reasoning capabilities of diffusion-based large language models (dLLMs) through reinforcement learning (RL) remains an open problem. The intractability of dLLMs likelihood function necessitates approximating the current, old,…

Machine Learning · Computer Science 2026-02-17 Xiaohang Tang , Rares Dolga , Sangwoong Yoon , Ilija Bogunovic

Diffusion-based decoding has recently emerged as an appealing alternative to autoregressive (AR) generation, offering the potential to update multiple tokens in parallel and reduce latency. However, diffusion vision language models (dVLMs)…

Computer Vision and Pattern Recognition · Computer Science 2026-04-01 Lunbin Zeng , Jingfeng Yao , Bencheng Liao , Hongyuan Tao , Wenyu Liu , Xinggang Wang

The diagnosis of pathological images is often limited by expert availability and regional disparities, highlighting the importance of automated diagnosis using Vision-Language Models (VLMs). Traditional multimodal models typically emphasize…

Computer Vision and Pattern Recognition · Computer Science 2025-04-21 Jianyu Wu , Hao Yang , Xinhua Zeng , Guibing He , Zhiyu Chen , Zihui Li , Xiaochuan Zhang , Yangyang Ma , Run Fang , Yang Liu

Reinforcement learning (RL) has emerged as an effective post-training paradigm for enhancing the reasoning capabilities of multimodal large language model (MLLM). However, current RL pipelines often suffer from training inefficiencies…

Machine Learning · Computer Science 2026-03-04 Linghao Zhu , Yiran Guan , Dingkang Liang , Jianzhong Ju , Zhenbo Luo , Bin Qin , Jian Luan , Yuliang Liu , Xiang Bai

Diffusion Models (DMs), as a leading class of generative models, offer key advantages for reinforcement learning (RL), including multi-modal expressiveness, stable training, and trajectory-level planning. This survey delivers a…

Machine Learning · Computer Science 2025-10-15 Changfu Xu , Jianxiong Guo , Yuzhu Liang , Haiyang Huang , Haodong Zou , Xi Zheng , Shui Yu , Xiaowen Chu , Jiannong Cao , Tian Wang

Vision-Language-Action (VLA) models aim to unify perception, language understanding, and action generation, offering strong cross-task and cross-scene generalization with broad impact on embodied AI. However, current VLA models often lack…

Computer Vision and Pattern Recognition · Computer Science 2025-10-03 Angen Ye , Zeyu Zhang , Boyuan Wang , Xiaofeng Wang , Dapeng Zhang , Zheng Zhu

The autonomous driving community is increasingly focused on addressing the challenges posed by out-of-distribution (OOD) driving scenarios. A dominant research trend seeks to enhance end-to-end (E2E) driving systems by integrating…

Computer Vision and Pattern Recognition · Computer Science 2025-12-05 Yingzi Ma , Yulong Cao , Wenhao Ding , Shuibai Zhang , Yan Wang , Boris Ivanovic , Ming Jiang , Marco Pavone , Chaowei Xiao

The rapid emergence of diverse large language models (LLMs) has spurred the development of LLM routers that assign user queries to the most suitable model. However, existing LLM routers typically perform a single-round, one-to-one mapping…

Computation and Language · Computer Science 2025-10-27 Haozhen Zhang , Tao Feng , Jiaxuan You