English
Related papers

Related papers: Stable Language Guidance for Vision-Language-Actio…

200 papers

Measurement of task progress through explicit, actionable milestones is critical for robust robotic manipulation. This progress awareness enables a model to ground its current task status, anticipate verifiable intermediate states, and…

Robotics · Computer Science 2026-03-11 Tingjun Dai , Mingfei Han , Tingwen Du , Zhiheng Liu , Zhihui Li , Salman Khan , Jun Yu , Xiaojun Chang

Large Language Models (LLMs) often struggle with problems that require multi-step reasoning. For small-scale open-source models, Reinforcement Learning with Verifiable Rewards (RLVR) fails when correct solutions are rarely sampled even…

Computation and Language · Computer Science 2026-03-02 Yihe Deng , I-Hung Hsu , Jun Yan , Zifeng Wang , Rujun Han , Gufeng Zhang , Yanfei Chen , Wei Wang , Tomas Pfister , Chen-Yu Lee

Multimodal recommendation enhances accuracy by leveraging visual and textual signals, and its success largely depends on learning high-quality cross-modal representations. Recent advances in Large Vision-Language Models (LVLMs) offer…

Information Retrieval · Computer Science 2026-04-28 Zhongtao Rao , Peilin Zhou , Dading Chong , Zhiwei Chen , Shoujin Wang , Nan Tang

Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for generalist robotic control. Built upon vision-language model (VLM) architectures, VLAs predict actions conditioned on visual observations and language…

Robotics · Computer Science 2026-05-26 Weikang Qiu , Huashuo Lei , Tinglin Huang , Rex Ying

Vision-Language-Action (VLA) models aim to unify perception, language understanding, and action generation, offering strong cross-task and cross-scene generalization with broad impact on embodied AI. However, current VLA models often lack…

Computer Vision and Pattern Recognition · Computer Science 2025-10-03 Angen Ye , Zeyu Zhang , Boyuan Wang , Xiaofeng Wang , Dapeng Zhang , Zheng Zhu

Vision-Language Models (VLMs) achieve strong cross-modal performance, yet recent evidence suggests they over-rely on textual descriptions while under-utilizing visual evidence -- a phenomenon termed ``text shortcut learning.'' We propose an…

Computer Vision and Pattern Recognition · Computer Science 2026-04-21 Lijie Zhou

The success of vision-language models is primarily attributed to effective alignment across modalities such as vision and language. However, modality gaps persist in existing alignment algorithms and appear necessary for human perception as…

Computer Vision and Pattern Recognition · Computer Science 2026-04-28 Hanqi Yan , Xiangxiang Cui , Lu Yin , Jindong Gu , Paul Pu Liang , Yulan He , Yifei Wang

Large language models (LLMs) achieve strong reasoning performance by allocating substantial computation at inference time, often generating long and verbose reasoning traces. While recent work on efficient reasoning reduces this overhead…

Computation and Language · Computer Science 2026-04-28 Han Wang , Xiaodong Yu , Jialian Wu , Jiang Liu , Ximeng Sun , Mohit Bansal , Zicheng Liu

Recent studies reveal that integrating new modalities into Large Language Models (LLMs), such as Vision-Language Models (VLMs), creates a new attack surface that bypasses existing safety training techniques like Supervised Fine-tuning (SFT)…

Computation and Language · Computer Science 2025-10-15 Trishna Chakraborty , Erfan Shayegani , Zikui Cai , Nael Abu-Ghazaleh , M. Salman Asif , Yue Dong , Amit K. Roy-Chowdhury , Chengyu Song

Vision-Language-Action (VLA) models advance robotic control via strong visual-linguistic priors. However, existing VLAs predominantly frame pretraining as supervised behavior cloning, overlooking the fundamental nature of robot learning as…

Artificial Intelligence · Computer Science 2026-05-01 Yang Zhang , Jiangyuan Zhao , Chenyou Fan , Fangzheng Yan , Tian Li , Haitong Tang , Sen Fu , Xuan'er Wu , Qizhen Weng , Weinan Zhang , Xiu Li , Chi Zhang , Chenjia Bai , Xuelong Li

Recently, some studies have integrated Multimodal Large Language Models into robotic manipulation, constructing vision-language-action models (VLAs) to interpret multimodal information and predict SE(3) poses. While VLAs have shown…

Computer Vision and Pattern Recognition · Computer Science 2025-03-20 Chenxuan Li , Jiaming Liu , Guanqun Wang , Xiaoqi Li , Sixiang Chen , Liang Heng , Chuyan Xiong , Jiaxin Ge , Renrui Zhang , Kaichen Zhou , Shanghang Zhang

This study investigates the spatial reasoning capabilities of vision-language models (VLMs) through Chain-of-Thought (CoT) prompting and reinforcement learning. We begin by evaluating the impact of different prompting strategies and find…

Computer Vision and Pattern Recognition · Computer Science 2025-07-21 Binbin Ji , Siddharth Agrawal , Qiance Tang , Yvonne Wu

Despite their strong performance in embodied tasks, recent Vision-Language-Action (VLA) models remain highly fragile under multimodal perturbations, where visual corruption and linguistic noise jointly induce distribution shifts that…

Robotics · Computer Science 2026-04-15 Yuhan Xie , Yuping Yan , Yunqi Zhao , Handing Wang , Yaochu Jin

Recent advances in reinforcement learning from human feedback (RLHF) and preference optimization have substantially improved the usability, coherence, and safety of large language models. However, recurring behaviors such as performative…

Artificial Intelligence · Computer Science 2026-05-13 William Parris

Multimodal Large Reasoning Models (MLRMs) have achieved remarkable strides in visual reasoning through test time compute scaling, yet long chain reasoning remains prone to hallucinations. We identify a concerning phenomenon termed the…

Artificial Intelligence · Computer Science 2026-05-29 Zhe Qian , Yanbiao Ma , Zhuohan Ouyang , Zhonghua Wang , Zhongxing Xu , Fei Luo , Xinyu Liu , Zongyuan Ge , Yike Guo , Jungong Han

Despite impressive advancements in Visual-Language Models (VLMs) for multi-modal tasks, their reliance on RGB inputs limits precise spatial understanding. Existing methods for integrating spatial cues, such as point clouds or depth, either…

Computer Vision and Pattern Recognition · Computer Science 2025-10-27 Yang Liu , Ming Ma , Xiaomin Yu , Pengxiang Ding , Han Zhao , Mingyang Sun , Siteng Huang , Donglin Wang

Vision-Language-Action (VLA) models inherit strong priors from pretrained Vision-Language Models (VLMs), but naive fine-tuning often disrupts these representations and harms generalization. Existing fixes -- freezing modules or applying…

Computer Vision and Pattern Recognition · Computer Science 2025-11-26 Chengyue Huang , Mellon M. Zhang , Robert Azarcon , Glen Chou , Zsolt Kira

Vision-Language-Action (VLA) models have made substantial progress by leveraging the robust capabilities of Visual Language Models (VLMs). However, VLMs' significant parameter size and autoregressive (AR) decoding nature impose considerable…

Machine Learning · Computer Science 2025-09-23 Songsheng Wang , Rucheng Yu , Zhihang Yuan , Chao Yu , Feng Gao , Yu Wang , Derek F. Wong

While large language models (LLMs) demonstrate strong reasoning capabilities utilizing reinforcement learning (RL) with verifiable reward, whether large vision-language models (VLMs) can directly inherit such capabilities through similar…

Artificial Intelligence · Computer Science 2025-05-27 Tianle Li , Jihai Zhang , Yongming Rao , Yu Cheng

Recent high-capacity vision-language-action (VLA) models have demonstrated impressive performance on a range of robotic manipulation tasks by imitating human demonstrations. However, exploiting offline data with limited visited states will…

Robotics · Computer Science 2025-05-27 Guanxing Lu , Wenkai Guo , Chubin Zhang , Yuheng Zhou , Haonan Jiang , Zifeng Gao , Yansong Tang , Ziwei Wang