English
Related papers

Related papers: Mind to Hand: Purposeful Robotic Control via Embod…

200 papers

Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. For example, in a search and rescue mission, a legged robot could climb over debris, crawl through gaps, and…

Robotics · Computer Science 2024-07-04 Annie S. Chen , Alec M. Lessing , Andy Tang , Govind Chada , Laura Smith , Sergey Levine , Chelsea Finn

Current research on Vision-Language-Action (VLA) models predominantly focuses on enhancing generalization through established reasoning techniques. While effective, these improvements invariably increase computational complexity and…

Computer Vision and Pattern Recognition · Computer Science 2026-03-06 Riccardo Andrea Izzo , Gianluca Bardaro , Matteo Matteucci

Autonomous driving has long relied on modular "Perception-Decision-Action" pipelines, where hand-crafted interfaces and rule-based components often break down in complex or long-tailed scenarios. Their cascaded design further propagates…

Recent advances in vision-language models (VLMs) have enabled instruction-conditioned robotic systems with improved generalization. However, most existing work focuses on reactive System 1 policies, underutilizing VLMs' strengths in…

Robotics · Computer Science 2025-10-30 Songhao Han , Boxiang Qiu , Yue Liao , Siyuan Huang , Chen Gao , Shuicheng Yan , Si Liu

Recent advances in embodied AI highlight the potential of vision language models (VLMs) as agents capable of perception, reasoning, and interaction in complex environments. However, top-performing systems rely on large-scale models that are…

Vision-Language-Action (VLA) models advance robotic control via strong visual-linguistic priors. However, existing VLAs predominantly frame pretraining as supervised behavior cloning, overlooking the fundamental nature of robot learning as…

Artificial Intelligence · Computer Science 2026-05-01 Yang Zhang , Jiangyuan Zhao , Chenyou Fan , Fangzheng Yan , Tian Li , Haitong Tang , Sen Fu , Xuan'er Wu , Qizhen Weng , Weinan Zhang , Xiu Li , Chi Zhang , Chenjia Bai , Xuelong Li

Despite their success, current training pipelines for reasoning VLMs focus on a limited range of tasks, such as mathematical and logical reasoning. As a result, these models face difficulties in generalizing their reasoning capabilities to…

Computer Vision and Pattern Recognition · Computer Science 2025-08-19 Yuheng Zha , Kun Zhou , Yujia Wu , Yushu Wang , Jie Feng , Zhi Xu , Shibo Hao , Zhengzhong Liu , Eric P. Xing , Zhiting Hu

Robotic manipulation in open-world environments requires reasoning across semantics, geometry, and long-horizon action dynamics. Existing hierarchical Vision-Language-Action (VLA) frameworks typically use 2D representations to connect…

Autonomous driving has progressed from modular pipelines toward end-to-end unification, and Vision-Language-Action (VLA) models are a natural extension of this journey beyond Vision-to-Action (VA). In practice, driving VLAs have often…

While end-to-end Vision-Language-Action (VLA) models offer a promising paradigm for robotic manipulation, fine-tuning them on narrow control data often compromises the profound reasoning capabilities inherited from their base…

Computer Vision and Pattern Recognition · Computer Science 2026-05-12 Tianshuo Yang , Guanyu Chen , Yutian Chen , Zhixuan Liang , Yitian Liu , Zanxin Chen , Chunpu Xu , Haotian Liang , Jiangmiao Pang , Yao Mu , Ping Luo

Vision-Language-Action (VLA) models have emerged as a powerful paradigm for embodied intelligence, enabling robots to perform tasks based on natural language instructions and current visual input. However, existing VLA models struggle with…

This paper introduces CognitiveDrone, a novel Vision-Language-Action (VLA) model tailored for complex Unmanned Aerial Vehicles (UAVs) tasks that demand advanced cognitive abilities. Trained on a dataset comprising over 8,000 simulated…

While Vision-Language-Action (VLA) models have revolutionized autonomous driving by unifying perception and planning, their reliance on explicit textual Chain-of-Thought (CoT) leads to semantic-perceptual decoupling and perceptual-symbolic…

Computer Vision and Pattern Recognition · Computer Science 2026-03-13 Yuechen Luo , Fang Li , Shaoqing Xu , Yang Ji , Zehan Zhang , Bing Wang , Yuannan Shen , Jianwei Cui , Long Chen , Guang Chen , Hangjun Ye , Zhi-Xin Yang , Fuxi Wen

The important manifestation of robot intelligence is the ability to naturally interact and autonomously make decisions. Traditional approaches to robot control often compartmentalize perception, planning, and decision-making, simplifying…

Robotics · Computer Science 2025-02-05 Pengxiang Ding , Han Zhao , Wenjie Zhang , Wenxuan Song , Min Zhang , Siteng Huang , Ningxi Yang , Donglin Wang

Future robotic systems operating in real-world environments will require on-board embodied intelligence without continuous cloud connection, balancing capabilities with constraints on computational power and memory. This work presents an…

Robotics · Computer Science 2025-09-03 Liam Boyle , Nicolas Baumann , Paviththiren Sivasothilingam , Michele Magno , Luca Benini

Vision-Language-Action (VLA) models have recently become highly prominent in the field of robotics. Leveraging vision-language foundation models trained on large-scale internet data, the VLA model can generate robotic actions directly from…

Robotics · Computer Science 2025-05-19 Wei Zhao , Gongsheng Li , Zhefei Gong , Pengxiang Ding , Han Zhao , Donglin Wang

Vision-Language-Action Models (VLAs) inherit their visual and linguistic capabilities from Vision-Language Models (VLMs), yet most VLAs are built from off-the-shelf VLMs that are not adapted to the embodied domain, limiting their downstream…

Computer Vision and Pattern Recognition · Computer Science 2026-04-23 Yiyang Du , Zhanqiu Guo , Xin Ye , Liu Ren , Chenyan Xiong

Multimodal reasoning stands as a pivotal capability for large vision-language models (LVLMs). The integration with Domain-Specific Languages (DSL), offering precise visual representations, equips these models with the opportunity to execute…

Computation and Language · Computer Science 2024-02-22 Xueliang Zhao , Xinting Huang , Tingchen Fu , Qintong Li , Shansan Gong , Lemao Liu , Wei Bi , Lingpeng Kong

Vision Language Models (VLMs) have recently been leveraged to generate robotic actions, forming Vision-Language-Action (VLA) models. However, directly adapting a pretrained VLM for robotic control remains challenging, particularly when…

Vision language models (VLMs) exhibit vast knowledge of the physical world, including intuition of physical and spatial properties, affordances, and motion. With fine-tuning, VLMs can also natively produce robot trajectories. We demonstrate…

Robotics · Computer Science 2025-05-16 William Xie , Max Conway , Yutong Zhang , Nikolaus Correll