English
Related papers

Related papers: From Spatial to Actions: Grounding Vision-Language…

200 papers

Vision-Language-Action (VLA) models drive next-generation autonomous systems, but training them requires scalable, high-quality annotations from complex environments. Current cloud pipelines rely on generic vision-language models (VLMs)…

Computer Vision and Pattern Recognition · Computer Science 2026-04-21 Kangan Qian , ChuChu Xie , Yang Zhong , Jingrui Pang , Siwen Jiao , Sicong Jiang , Zilin Huang , Yunlong Wang , Kun Jiang , Mengmeng Yang , Hao Ye , Guanghao Zhang , Hangjun Ye , Guang Chen , Long Chen , Diange Yang

We propose Avi, a novel 3D Vision-Language-Action (VLA) architecture that reframes robotic action generation as a problem of 3D perception and spatial reasoning, rather than low-level policy learning. While existing VLA models primarily…

Robotics · Computer Science 2025-10-28 Harris Song , Long Le

Latent Action Models (LAMs) enable Vision- Language-Action (VLA) systems to learn semantic action representations from large-scale unannotated data. Yet, we identify two bottlenecks of LAMs: 1) the commonly adopted end-to-end trained image…

Computer Vision and Pattern Recognition · Computer Science 2026-03-12 Zhejia Cai , Yandan Yang , Xinyuan Chang , Shiyi Liang , Ronghan Chen , Feng Xiong , Mu Xu , Ruqi Huang

Vision-Language-Action (VLA) models built on pretrained Vision-Language Models (VLMs) show strong potential but are limited in practicality due to their large parameter counts. To mitigate this issue, using a lightweight VLM has been…

Computer Vision and Pattern Recognition · Computer Science 2025-12-02 Chaojun Ni , Cheng Chen , Xiaofeng Wang , Zheng Zhu , Wenzhao Zheng , Boyuan Wang , Tianrun Chen , Guosheng Zhao , Haoyun Li , Zhehao Dong , Qiang Zhang , Yun Ye , Yang Wang , Guan Huang , Wenjun Mei

Recent advances in vision-language-action (VLA) models have shown promise in integrating image generation with action prediction to improve generalization and reasoning in robot manipulation. However, existing methods are limited to…

Computer Vision and Pattern Recognition · Computer Science 2025-08-27 Wenyao Zhang , Hongsi Liu , Zekun Qi , Yunnan Wang , Xinqiang Yu , Jiazhao Zhang , Runpei Dong , Jiawei He , Fan Lu , He Wang , Zhizheng Zhang , Li Yi , Wenjun Zeng , Xin Jin

Vision-Language-Action (VLA) models have emerged as a promising approach for enabling robots to follow language instructions and predict corresponding actions. However, current VLA models mainly rely on 2D visual inputs, neglecting the rich…

Robotics · Computer Science 2025-08-14 Lin Sun , Bin Xie , Yingfei Liu , Hao Shi , Tiancai Wang , Jiale Cao

Vision-Language-Action models have achieved remarkable progress in robotic manipulation, yet they suffer from a critical limitation: a lack of 3D scene understanding. This deficiency manifests as three intertwined challenges: weak…

Robotics · Computer Science 2026-05-29 Zhongyu Xia , Yousen Tang , Bingqing Wei , Yongtao Wang

Vision-Language-Action models have emerged as a promising paradigm for robotic manipulation by unifying perception, language grounding, and action generation. However, they often struggle in scenarios requiring precise spatial…

Computer Vision and Pattern Recognition · Computer Science 2026-05-15 Tao Lin , Yuxin Du , Jiting Liu , Nuobei Zhu , Yunhe Li , Yuqian Fu , Yinxinyu Chen , Hongyi Cai , Zewei Ye , Bing Cheng , Kai Ye , Yiran Mao , Yilei Zhong , MingKang Dong , Junchi Yan , Gen Li , Bo Zhao

While Vision-Language-Action (VLA) models have revolutionized autonomous driving by unifying perception and planning, their reliance on explicit textual Chain-of-Thought (CoT) leads to semantic-perceptual decoupling and perceptual-symbolic…

Computer Vision and Pattern Recognition · Computer Science 2026-03-13 Yuechen Luo , Fang Li , Shaoqing Xu , Yang Ji , Zehan Zhang , Bing Wang , Yuannan Shen , Jianwei Cui , Long Chen , Guang Chen , Hangjun Ye , Zhi-Xin Yang , Fuxi Wen

We present FoundAtion-model-guided decoupled LoCO-maNipulation visuomotor policies (FALCON), a framework for loco-manipulation that combines modular diffusion policies with a vision-language foundation model as the coordinator. Our approach…

Robotics · Computer Science 2025-12-05 Chengyang He , Ge Sun , Yue Bai , Junkai Lu , Jiadong Zhao , Guillaume Sartoretti

While large language models have become the prevailing approach for agentic reasoning and planning, their success in symbolic domains does not readily translate to the physical world. Spatial intelligence, the ability to perceive 3D…

Machine Learning · Computer Science 2026-02-03 Gloria Felicia , Nolan Bryant , Handi Putra , Ayaan Gazali , Eliel Lobo , Esteban Rojas

Visual-Language-Action (VLA) models represent a paradigm shift in embodied AI, yet existing frameworks often struggle with imprecise spatial perception, suboptimal multimodal fusion, and instability in reinforcement learning. To bridge…

Robotics · Computer Science 2026-04-27 Haoxiang Jie , Yaoyuan Yan , Xiangyu Wei , Kailin Wang , Hongjie Yan , Zhiyou Heng , Daocheng Chen

Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency because they entangle high-level perception with sparse, embodiment-specific action supervision. Since these models typically rely…

Computer Vision and Pattern Recognition · Computer Science 2026-05-19 Haitao Lin , Hanyang Yu , Jingshun Huang , He Zhang , Yonggen Ling , Ping Tan , Xiangyang Xue , Yanwei Fu

We introduce FALCON, a unified self-supervised video pretraining approach for UAV action recognition from raw RGB aerial footage, requiring no additional preprocessing at inference. UAV videos exhibit severe spatial imbalance: large,…

Computer Vision and Pattern Recognition · Computer Science 2026-03-09 Ruiqi Xian , Xiyang Wu , Tianrui Guan , Xijun Wang , Boqing Gong , Dinesh Manocha

Recent advances in LVLMs have improved vision-language understanding, but they still struggle with spatial perception, limiting their ability to reason about complex 3D scenes. Unlike previous approaches that incorporate 3D representations…

Computer Vision and Pattern Recognition · Computer Science 2026-01-13 Jiahui Zhang , Yurui Chen , Yanpeng Zhou , Yueming Xu , Ze Huang , Jilin Mei , Junhui Chen , Yu-Jie Yuan , Xinyue Cai , Guowei Huang , Xingyue Quan , Hang Xu , Li Zhang

Vision-Language-Action (VLA) models extend vision-language models to embodied control by mapping natural-language instructions and visual observations to robot actions. Despite their capabilities, VLA systems face significant challenges due…

Robotics · Computer Science 2025-10-24 Weifan Guan , Qinghao Hu , Aosheng Li , Jian Cheng

Achieving generalization in robotic manipulation remains a critical challenge, particularly for unseen scenarios and novel tasks. Current Vision-Language-Action (VLA) models, while building on top of general Vision-Language Models (VLMs),…

Robotics · Computer Science 2026-04-07 Yifu Yuan , Haiqin Cui , Yibin Chen , Zibin Dong , Fei Ni , Longxin Kou , Jinyi Liu , Pengyi Li , Yan Zheng , Jianye Hao

The rapid progress of auto-regressive vision-language models (VLMs) has inspired growing interest in vision-language-action models (VLA) for robotic manipulation. Recently, masked diffusion models, a paradigm distinct from autoregressive…

Robotics · Computer Science 2025-09-11 Yuqing Wen , Hebei Li , Kefan Gu , Yucheng Zhao , Tiancai Wang , Xiaoyan Sun

Vision-Language-Action (VLA) models have shown strong potential for general-purpose robotic manipulation by leveraging large pretrained vision-language backbones. However, most existing VLAs rely primarily on 2D visual representations,…

Robotics · Computer Science 2026-05-21 Shizhe Chen , Paul Pacaud , Cordelia Schmid

While significant research has focused on developing embodied reasoning capabilities using Vision-Language Models (VLMs) or integrating advanced VLMs into Vision-Language-Action (VLA) models for end-to-end robot control, few studies…

Computer Vision and Pattern Recognition · Computer Science 2026-01-28 Ganlin Yang , Tianyi Zhang , Haoran Hao , Weiyun Wang , Yibin Liu , Dehui Wang , Guanzhou Chen , Zijian Cai , Junting Chen , Weijie Su , Wengang Zhou , Yu Qiao , Jifeng Dai , Jiangmiao Pang , Gen Luo , Wenhai Wang , Yao Mu , Zhi Hou