English
Related papers

Related papers: VLS: Steering Pretrained Robot Policies via Vision…

200 papers

Recently, some studies have integrated Multimodal Large Language Models into robotic manipulation, constructing vision-language-action models (VLAs) to interpret multimodal information and predict SE(3) poses. While VLAs have shown…

Computer Vision and Pattern Recognition · Computer Science 2025-03-20 Chenxuan Li , Jiaming Liu , Guanqun Wang , Xiaoqi Li , Sixiang Chen , Liang Heng , Chuyan Xiong , Jiaxin Ge , Renrui Zhang , Kaichen Zhou , Shanghang Zhang

Flow-matching-based policies have recently emerged as a promising approach for learning-based robot manipulation, offering significant acceleration in action sampling compared to diffusion-based policies. However, conventional flow-matching…

Robotics · Computer Science 2025-10-03 Xuanran Zhai , Qianyou Zhao , Qiaojun Yu , Ce Hao

Diffusion policies have demonstrated strong performance in generative modeling, making them promising for robotic manipulation guided by natural language instructions. However, generalizing language-conditioned diffusion policies to…

Robotics · Computer Science 2025-08-20 Ce Hao , Kelvin Lin , Zhiwei Xue , Siyuan Luo , Harold Soh

Fine-tuning pre-trained robot policies with reinforcement learning (RL) often inherits the bottlenecks introduced by pre-training with behavioral cloning (BC), which produces narrow action distributions that lack the coverage necessary for…

Robotics · Computer Science 2026-05-13 Matthew M. Hong , Jesse Zhang , Anusha Nagabandi , Abhishek Gupta

Diffusion policies have emerged as a mainstream paradigm for building vision-language-action (VLA) models. Although they demonstrate strong robot control capabilities, their training efficiency remains suboptimal. In this work, we identify…

Robotics · Computer Science 2025-05-19 Zibin Dong , Yicheng Liu , Yinchuan Li , Hang Zhao , Jianye Hao

Diffusion policy has demonstrated promising performance in the field of robotic manipulation. However, its effectiveness has been primarily limited in short-horizon tasks, and its performance significantly degrades in the presence of image…

Robotics · Computer Science 2025-07-08 Kefeng Huang , Tingguang Li , Yuzhen Liu , Zhe Zhang , Jiankun Wang , Lei Han

Recent progress in vision language foundation models has shown their ability to understand multimodal data and resolve complicated vision language tasks, including robotics manipulation. We seek a straightforward way of making use of…

Vision-Language-Action (VLA) models aim to unify perception, language understanding, and action generation, offering strong cross-task and cross-scene generalization with broad impact on embodied AI. However, current VLA models often lack…

Computer Vision and Pattern Recognition · Computer Science 2025-10-03 Angen Ye , Zeyu Zhang , Boyuan Wang , Xiaofeng Wang , Dapeng Zhang , Zheng Zhu

While significant research has focused on developing embodied reasoning capabilities using Vision-Language Models (VLMs) or integrating advanced VLMs into Vision-Language-Action (VLA) models for end-to-end robot control, few studies…

Computer Vision and Pattern Recognition · Computer Science 2026-01-28 Ganlin Yang , Tianyi Zhang , Haoran Hao , Weiyun Wang , Yibin Liu , Dehui Wang , Guanzhou Chen , Zijian Cai , Junting Chen , Weijie Su , Wengang Zhou , Yu Qiao , Jifeng Dai , Jiangmiao Pang , Gen Luo , Wenhai Wang , Yao Mu , Zhi Hou

Vision-Language-Action (VLA) models have shown promise in robot manipulation but often struggle to generalize to new instructions or complex multi-task scenarios. We identify a critical pathology in current training paradigms where…

Artificial Intelligence · Computer Science 2026-05-14 Shijie Lian , Bin Yu , Xiaopeng Lin , Laurence T. Yang , Zhaolong Shen , Changti Wu , Yuzhuo Miao , Cong Huang , Kai Chen

A fundamental challenge in autonomous driving is the integration of high-level, semantic reasoning for long-tail events with low-level, reactive control for robust driving. While large vision-language models (VLMs) trained on web-scale data…

Vision--Language--Action (VLA) models that encode actions using a discrete tokenization scheme are increasingly adopted for robotic manipulation, but existing decoding paradigms remain fundamentally limited. Whether actions are decoded…

Robotics · Computer Science 2026-04-08 Jiayi Chen , Wenxuan Song , Shuai Chen , Jingbo Wang , Zhijun Li , Haoang Li

Vision-Language-Action (VLA) models remain brittle in long-horizon, contact-rich manipulation because success-only imitation provides little supervision for execution drift, while failed rollouts are often discarded. We introduce RePO-VLA,…

Long-horizon robotic manipulation requires plans that are both logically coherent and geometrically grounded. Existing Vision-Language-Action policies usually hide planning in latent states or expose only one modality: text-only…

Artificial Intelligence · Computer Science 2026-05-04 Jinkun Liu , Haohan Chi , Lingfeng Zhang , Yifan Xie , YuAn Wang , Long Chen , Hangjun Ye , Xiaoshuai Hao , Wenbo Ding

Leveraging diverse robotic data for pretraining remains a critical challenge. Existing methods typically model the dataset's action distribution using simple observations as inputs. However, these inputs are often incomplete, resulting in a…

Computer Vision and Pattern Recognition · Computer Science 2025-11-19 Jiahui Zhang , Yurui Chen , Yueming Xu , Ze Huang , Yanpeng Zhou , Yu-Jie Yuan , Xinyue Cai , Guowei Huang , Xingyue Quan , Hang Xu , Li Zhang

Flow matching policies learn continuous velocity fields that transport noise to actions, enabling fast deterministic inference for robot manipulation. However, standard training optimizes a pointwise velocity objective while inference…

Robotics · Computer Science 2026-05-12 Riad Ahmed , Sujosh Nag , Moniruzzaman Akash , Mostafa Hussein , Momotaz Begum

Vision-Language Navigation in Continuous Environments (VLNCE), where an agent follows instructions and moves freely to reach a destination, is a key research problem in embodied AI. However, most existing approaches are sensitive to…

Computer Vision and Pattern Recognition · Computer Science 2026-02-23 Josh Qixuan Sun , Huaiyuan Weng , Xiaoying Xing , Chul Min Yeum , Mark Crowley

Learning a generalizable bimanual manipulation policy is extremely challenging for embodied agents due to the large action space and the need for coordinated arm movements. Existing approaches rely on Vision-Language-Action (VLA) models to…

Computer Vision and Pattern Recognition · Computer Science 2025-06-02 Chenyou Fan , Fangzheng Yan , Chenjia Bai , Jiepeng Wang , Chi Zhang , Zhen Wang , Xuelong Li

In visuomotor policy learning, the control policy for the robotic agent is derived directly from visual inputs. The typical approach, where a policy and vision encoder are trained jointly from scratch, generalizes poorly to novel visual…

Robotics · Computer Science 2025-09-17 Scott Jones , Liyou Zhou , Sebastian W. Pattinson

Vision-Language-Action (VLA) models aim to predict robotic actions based on visual observations and language instructions. Existing approaches require fine-tuning pre-trained visionlanguage models (VLMs) as visual and language features are…