English
Related papers

Related papers: Learning Vision-Language-Action World Models for A…

200 papers

Vision-Language-Action (VLA) models with integrated reasoning have been proposed for end-to-end autonomous driving, assuming a tight coupling between reasoning and trajectory generation. However, the robustness of such systems under…

Cryptography and Security · Computer Science 2026-05-29 Mohammadreza Teymoorianfard , Jean-Philippe Monteuuis , Jonathan Petit , Amir Houmansadr

World models have attracted increasing attention in autonomous driving for their ability to forecast potential future scenarios. In this paper, we propose BEVWorld, a novel framework that transforms multimodal sensor inputs into a unified…

Computer Vision and Pattern Recognition · Computer Science 2025-05-01 Yumeng Zhang , Shi Gong , Kaixin Xiong , Xiaoqing Ye , Xiaofan Li , Xiao Tan , Fan Wang , Jizhou Huang , Hua Wu , Haifeng Wang

Vision-Language-Action models (VLAs) are emerging as powerful tools for learning generalizable visuomotor control policies. However, current VLAs are mostly trained on large-scale image-text-action data and remain limited in two key ways:…

Computer Vision and Pattern Recognition · Computer Science 2026-03-24 Wenqi Liang , Gan Sun , Yao He , Jiahua Dong , Suyan Dai , Ivan Laptev , Salman Khan , Yang Cong

Latent Action Models (LAMs) have rapidly gained traction as an important component in the pre-training pipelines of leading Vision-Language-Action models. However, they fail when observations contain action-correlated distractors, often…

Vision-Language-Action (VLA) models have emerged as a generalist robotic agent. However, existing VLAs are hindered by excessive parameter scales, prohibitive pre-training requirements, and limited applicability to diverse embodiments. To…

Vision-Language-Action (VLA) models commonly adapt pretrained Vision-Language Models (VLMs) to robot control by mapping visual observations and language instructions to continuous actions. Existing approaches typically take an…

Computer Vision and Pattern Recognition · Computer Science 2026-05-26 Xuan Wang , Yinan Wu , Haoran Duan , Jungong Han

Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments. In…

World models aim to learn action-controlled future prediction and have proven essential for the development of intelligent agents. However, most existing world models rely heavily on substantial action-labeled data and costly training,…

Artificial Intelligence · Computer Science 2025-06-03 Shenyuan Gao , Siyuan Zhou , Yilun Du , Jun Zhang , Chuang Gan

Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness, typically relying on hand-crafted heuristics for task termination. This limitation is particularly severe in long-horizon tasks involving…

Robotics · Computer Science 2026-03-31 Hongyu Yan , Qiwei Li , Jiaolong Yang , Yadong Mu

Vision-Language-Action (VLA) models have demonstrated strong multi-modal reasoning capabilities, enabling direct action generation from visual perception and language instructions in an end-to-end manner. However, their substantial…

Robotics · Computer Science 2025-10-22 Siyu Xu , Yunke Wang , Chenghao Xia , Dihao Zhu , Tao Huang , Chang Xu

Recent advances in Vision-Language-Action (VLA) models have opened new avenues for robot manipulation, yet existing methods exhibit limited efficiency and a lack of high-level knowledge and spatial awareness. To address these challenges, we…

Vision-Language-Action (VLA) models are a powerful paradigm for generalist robotic control. However, their high computational cost and limited control frequency hinder real-time robotic manipulation, especially when large vision-language…

Robotics · Computer Science 2026-05-29 Ye Li , Huanan Liu , Kangye Ji , Yuan Meng , Jiajun Fan , Yuansong Wang , Shiyu Qin , Chenglei Wu , Shu-Tao Xia , Zhi Wang

We propose a standalone autoregressive (AR) Action Expert that generates actions as a continuous causal sequence while conditioning on refreshable vision-language prefixes. In contrast to existing Vision-Language-Action (VLA) models and…

Recently, world models have been incorporated into the autonomous driving systems to improve the planning reliability. Existing approaches typically predict future states through appearance generation or deterministic regression, which…

Computer Vision and Pattern Recognition · Computer Science 2026-05-05 Xiaolu Liu , Yicong Li , Song Wang , Junbo Chen , Angela Yao , Jianke Zhu

Vision-Language-Action (VLA)-based driving systems represent a significant paradigm shift in autonomous driving since, by combining traffic scene understanding, linguistic interpretation, and action generation, these systems enable more…

Robotics · Computer Science 2026-03-19 Gerhard Yu , Fuyuki Ishikawa , Oluwafemi Odu , Alvine Boaye Belle

Vision-language models (VLMs) have recently emerged as powerful representation learning systems that align visual observations with natural language concepts, offering new opportunities for semantic reasoning in safety-critical autonomous…

Computer Vision and Pattern Recognition · Computer Science 2026-02-19 Ross Greer , Maitrayee Keskar , Angel Martinez-Sanchez , Parthib Roy , Shashank Shriram , Mohan Trivedi

Vision-language-action (VLA) models aim to understand natural language instructions and visual observations and to execute corresponding actions as an embodied agent. Recent work integrates future images into the understanding-acting loop,…

Robotics · Computer Science 2026-03-26 Jiayi Chen , Wenxuan Song , Pengxiang Ding , Ziyang Zhou , Han Zhao , Feilong Tang , Donglin Wang , Haoang Li

Lifelong learning is critical for embodied agents in open-world environments, where reinforcement learning fine-tuning has emerged as an important paradigm to enable Vision-Language-Action (VLA) models to master dexterous manipulation…

Artificial Intelligence · Computer Science 2026-02-04 Qixin Zeng , Shuo Zhang , Hongyin Zhang , Renjie Wang , Han Zhao , Libang Zhao , Runze Li , Donglin Wang , Chao Huang

Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for generalist robotic control. Built upon vision-language model (VLM) architectures, VLAs predict actions conditioned on visual observations and language…

Robotics · Computer Science 2026-05-26 Weikang Qiu , Huashuo Lei , Tinglin Huang , Rex Ying

Recent advances in vision-language-action (VLA) models have motivated the extension of their capabilities to embodied settings, where reinforcement learning (RL) offers a principled way to optimize task success through interaction. However,…