中文
相关论文

相关论文: Reinforcing Multimodal Reasoning Against Visual De…

200 篇论文

Large Language Models (LLMs) demonstrate transformative potential, yet their reasoning remains inconsistent and unreliable. Reinforcement learning (RL)-based fine-tuning is a key mechanism for improvement, but its effectiveness is…

机器学习 · 计算机科学 2026-02-11 Pei-Chi Pan , Yingbin Liang , Sen Lin

Recent advances in large language models have significantly improved textual reasoning through the effective use of Chain-of-Thought (CoT) and reinforcement learning. However, extending these successes to vision-language tasks remains…

计算机视觉与模式识别 · 计算机科学 2025-05-27 Minheng Ni , Zhengyuan Yang , Linjie Li , Chung-Ching Lin , Kevin Lin , Wangmeng Zuo , Lijuan Wang

Generating grounded and trustworthy responses remains a key challenge for large language models (LLMs). While retrieval-augmented generation (RAG) with citation-based grounding holds promise, instruction-tuned models frequently fail even in…

计算与语言 · 计算机科学 2025-06-19 Shang Hong Sim , Tej Deep Pala , Vernon Toh , Hai Leong Chieu , Amir Zadeh , Chuan Li , Navonil Majumder , Soujanya Poria

Complex image restoration aims to recover high-quality images from inputs affected by multiple degradations such as blur, noise, rain, and compression artifacts. Recent restoration agents, powered by vision-language models and large…

计算机视觉与模式识别 · 计算机科学 2026-04-07 Jianglin Lu , Yuanwei Wu , Ziyi Zhao , Hongcheng Wang , Felix Jimenez , Abrar Majeedi , Yun Fu

Techniques that learn improved representations via offline data or self-supervised objectives have shown impressive results in traditional reinforcement learning (RL). Nevertheless, it is unclear how improved representation learning can…

计算与语言 · 计算机科学 2024-10-25 Vaskar Nath , Dylan Slack , Jeff Da , Yuntao Ma , Hugh Zhang , Spencer Whitehead , Sean Hendryx

Omnidirectional images (ODIs), with their 360{\deg} field of view, provide unparalleled spatial awareness for immersive applications like augmented reality and embodied AI. However, the capability of existing multi-modal large language…

计算机视觉与模式识别 · 计算机科学 2025-05-21 Xinshen Zhang , Zhen Ye , Xu Zheng

Reinforcement Learning with Verifiable Rewards (RLVR) enhances Large Language Model (LLM) reasoning but suffers from advantage collapse on ``hard samples'' where all rollouts fail. This lack of variance eliminates crucial learning signals.…

机器学习 · 计算机科学 2026-05-08 Xinyu Lu , Kaiqi Zhang , Jinglin Yang , Boxi Cao , Yaojie Lu , Hongyu Lin , Min He , Xianpei Han , Le Sun

Vision-Language Models (VLMs) frequently suffer from visual perception errors and hallucinations that compromise answer accuracy in complex reasoning tasks. Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising solution…

计算机视觉与模式识别 · 计算机科学 2026-05-05 Yin Zhang , Jiaxuan Zhao , Zonghan Wu , Zengxiang Li , Junfeng Fang , Kun Wang , Qingsong Wen , Yilei Shao

Recent progress in large-scale reinforcement learning (RL) has notably enhanced the reasoning capabilities of large language models (LLMs), especially in mathematical domains. However, current multimodal LLMs (MLLMs) for mathematical…

计算与语言 · 计算机科学 2025-07-04 Wenhao Shi , Zhiqiang Hu , Yi Bin , Yang Yang , See-Kiong Ng , Heng Tao Shen

Large Multimodal Models (LMMs), or Vision-Language Models (VLMs), have shown impressive capabilities in a wide range of visual tasks. However, they often struggle with fine-grained visual reasoning, failing to identify domain-specific…

计算机视觉与模式识别 · 计算机科学 2025-02-26 Yucheng Shi , Quanzheng Li , Jin Sun , Xiang Li , Ninghao Liu

Recent breakthroughs in reasoning models have markedly advanced the reasoning capabilities of large language models, particularly via training on tasks with verifiable rewards. Yet, a significant gap persists in their adaptation to real…

计算机视觉与模式识别 · 计算机科学 2025-10-28 Jiaao Yu , Shenwei Li , Mingjie Han , Yifei Yin , Wenzheng Song , Chenghao Jia , Man Lan

Multimodal Large Language Models (MLLMs) have increasingly localized and interleaved visual evidence for deliberative reasoning. Grounding-based approaches typically focus on regions of interest (RoIs) by injecting cropped image patches or…

计算机视觉与模式识别 · 计算机科学 2026-05-29 Guannan Lv , Ren Nie , Hongjian Dou , Tingting Gao

Vision-Language-Action (VLA) models aim to unify perception, language understanding, and action generation, offering strong cross-task and cross-scene generalization with broad impact on embodied AI. However, current VLA models often lack…

计算机视觉与模式识别 · 计算机科学 2025-10-03 Angen Ye , Zeyu Zhang , Boyuan Wang , Xiaofeng Wang , Dapeng Zhang , Zheng Zhu

Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced the reasoning capacity of Large Language Models (LLMs). However, RLVR solely relies on final answers as outcome rewards, neglecting the correctness of…

机器学习 · 计算机科学 2026-03-12 Sijia Cui , Pengyu Cheng , Jiajun Song , Yongbo Gai , Guojun Zhang , Zhechao Yu , Jianhe Lin , Xiaoxi Jiang , Guanjun Jiang

Multimodal large language models (MLLMs) have advanced perception across text, vision, and audio, yet they often struggle with structured cross-modal reasoning, particularly when integrating audio and visual signals. We introduce…

计算机视觉与模式识别 · 计算机科学 2025-06-20 Zhenghao Xing , Xiaowei Hu , Chi-Wing Fu , Wenhai Wang , Jifeng Dai , Pheng-Ann Heng

Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in vision-language answering tasks. Despite their strengths, these models often encounter challenges in achieving complex reasoning tasks such as…

人工智能 · 计算机科学 2025-11-11 Jinhao Chen , Zhen Yang , Jianxin Shi , Tianyu Wo , Jie Tang

Advances in large reasoning models have shown strong performance on complex reasoning tasks by scaling test-time compute through extended reasoning. However, recent studies observe that in vision-dependent tasks, extended textual reasoning…

计算机视觉与模式识别 · 计算机科学 2026-03-03 Soumya Suvra Ghosal , Youngeun Kim , Zhuowei Li , Ritwick Chaudhry , Linghan Xu , Hongjing Zhang , Jakub Zablocki , Yifan Xing , Qin Zhang

Self-correction is essential for solving complex reasoning problems in vision-language models (VLMs). However, existing reinforcement learning (RL) methods struggle to learn it, as effective self-correction behaviors emerge only rarely,…

计算机视觉与模式识别 · 计算机科学 2026-02-10 Yi Ding , Ziliang Qiu , Bolian Li , Ruqi Zhang

Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as a powerful paradigm for facilitating the self-improvement of large language models (LLMs), particularly in the domain of complex reasoning tasks. However,…

机器学习 · 计算机科学 2025-07-17 Ziru Liu , Cheng Gong , Xinyu Fu , Yaofang Liu , Ran Chen , Shoubo Hu , Suiyun Zhang , Rui Liu , Qingfu Zhang , Dandan Tu

Multimodal Large Language Models (MLLMs) have shown remarkable progress in visual reasoning and understanding tasks but still struggle to capture the complexity and subjectivity of human emotions. Existing approaches based on supervised…

人工智能 · 计算机科学 2026-03-02 Yiyang Fang , Wenke Huang , Pei Fu , Yihao Yang , Kehua Su , Zhenbo Luo , Jian Luan , Mang Ye