English
Related papers

Related papers: Compressor-VLA: Instruction-Guided Visual Token Co…

200 papers

Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often constrained by cost: billion-scale VLM backbones and iterative diffusion/flow-based action…

Recent Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language understanding tasks, yet their inference efficiency is often hampered by the large number of visual tokens, particularly in…

Computer Vision and Pattern Recognition · Computer Science 2026-04-21 Jiafei Song , Fengwei Zhou , Jin Qu , Wenjin Jason Li , Tong Wu , Gengjian Xue , Zhikang Zhao , Daomin Wei , Yichao Lu , Bailin Na

Leveraging temporal context is crucial for success in partially observable robotic tasks. However, prior work in behavior cloning has demonstrated inconsistent performance gains when using multi-frame observations. In this paper, we…

Robotics · Computer Science 2025-10-07 Huiwon Jang , Sihyun Yu , Heeseung Kwon , Hojin Jeon , Younggyo Seo , Jinwoo Shin

Since current Vision-Language-Action (VLA) systems suffer from limited spatial perception and the absence of memory throughout manipulation, we investigate visual anchors as a means to enhance spatial and temporal reasoning within VLA…

Robotics · Computer Science 2026-03-16 Juan Zhu , Zhanying Shao , Xiaoqi Li , Ethan Morgan , Jiadong Xu , Hongwei Fan , Hao Dong

Recent high-capacity vision-language-action (VLA) models have demonstrated impressive performance on a range of robotic manipulation tasks by imitating human demonstrations. However, exploiting offline data with limited visited states will…

Robotics · Computer Science 2025-05-27 Guanxing Lu , Wenkai Guo , Chubin Zhang , Yuheng Zhou , Haonan Jiang , Zifeng Gao , Yansong Tang , Ziwei Wang

Autoregressive vision-language-action (VLA) models have recently demonstrated strong capabilities in robotic manipulation. However, their core process of action tokenization often involves a trade-off between reconstruction fidelity and…

Computer Vision and Pattern Recognition · Computer Science 2025-12-09 Yicheng Liu , Shiduo Zhang , Zibin Dong , Baijun Ye , Tianyuan Yuan , Xiaopeng Yu , Linqi Yin , Chenhao Lu , Junhao Shi , Luca Jiang-Tao Yu , Liangtao Zheng , Tao Jiang , Jingjing Gong , Xipeng Qiu , Hang Zhao

Vision-Language-Action (VLA) models aim to control robots for manipulation from visual observations and natural-language instructions. However, existing hierarchical and autoregressive paradigms often introduce architectural overhead,…

Large Vision-Language Models (LVLMs) incur substantial inference costs due to the processing of a vast number of visual tokens. Existing methods typically struggle to model progressive visual token reduction as a multi-step decision process…

Computer Vision and Pattern Recognition · Computer Science 2026-03-17 Sihan Cao , Jianwei Zhang , Pengcheng Zheng , Jiaxin Yan , Caiyan Qin , Yalan Ye , Wei Dong , Peng Wang , Yang Yang , Chaoning Zhang

Vision-Language-Action (VLA) models have achieved significant breakthroughs by leveraging Large Vision Language Models (VLMs) to jointly interpret instructions and visual inputs. However, the substantial increase in visual tokens,…

Robotics · Computer Science 2026-02-25 Haosheng Li , Weixin Mao , Zihan Lan , Hongwei Xiong , Hongan Wang , Chenyang Si , Ziwei Liu , Xiaoming Deng , Hua Chen

While significant advancements have been made in compressed representations for text embeddings in large language models (LLMs), the compression of visual tokens in multi-modal LLMs (MLLMs) has remained a largely overlooked area. In this…

Computer Vision and Pattern Recognition · Computer Science 2024-11-19 Jieneng Chen , Luoxin Ye , Ju He , Zhao-Yang Wang , Daniel Khashabi , Alan Yuille

In this paper, we present LLaVA-Scissor, a training-free token compression strategy designed for video multimodal large language models. Previous methods mostly attempt to compress tokens based on attention scores, but fail to effectively…

Computer Vision and Pattern Recognition · Computer Science 2025-06-30 Boyuan Sun , Jiaxing Zhao , Xihan Wei , Qibin Hou

Vision-Language-Action models (VLAs) are emerging as powerful tools for learning generalizable visuomotor control policies. However, current VLAs are mostly trained on large-scale image-text-action data and remain limited in two key ways:…

Computer Vision and Pattern Recognition · Computer Science 2026-03-24 Wenqi Liang , Gan Sun , Yao He , Jiahua Dong , Suyan Dai , Ivan Laptev , Salman Khan , Yang Cong

Vision-Language-Action (VLA) models have shown remarkable potential in visuomotor control and instruction comprehension through end-to-end learning processes. However, current VLA models face significant challenges: they are slow during…

Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for generalist robotic control. Built upon vision-language model (VLM) architectures, VLAs predict actions conditioned on visual observations and language…

Robotics · Computer Science 2026-05-26 Weikang Qiu , Huashuo Lei , Tinglin Huang , Rex Ying

Vision-Language-Action (VLA) models have emerged as a powerful paradigm for robotic manipulation. However, existing post-training methods face a dilemma between stability and exploration: Supervised Fine-Tuning (SFT) is constrained by…

Robotics · Computer Science 2026-03-17 Jiashun Li , Xiaoyu Shi , Hong Xie , Mingsheng Shang , Yun Lu

While vision-language-action (VLA) models have shown great promise for robot manipulation, their deployment on rigid industrial robots remains challenging due to the inherent trade-off between compliance and responsiveness. Standard…

Robotics · Computer Science 2026-03-18 Johannes Hechtl , Philipp Schmitt , Georg von Wichert , Wolfram Burgard

Vision-Language-Action (VLA) models leverage pretrained vision-language models (VLMs) to couple perception with robotic control, offering a promising path toward general-purpose embodied intelligence. However, current SOTA VLAs are…

Robotics · Computer Science 2025-10-10 Yandu Chen , Kefan Gu , Yuqing Wen , Yucheng Zhao , Tiancai Wang , Liqiang Nie

Real-time inference of vision-language-action (VLA) models is essential for robotic control. While visual token pruning has shown strong potential for accelerating inference, most existing methods mainly base pruning decisions on…

Computer Vision and Pattern Recognition · Computer Science 2026-05-29 Shilin Ma , Chubin Zhang , Changyuan Wang , Yuji Wang , Yue Wu , Zixuan Wang , Jingqi Tian , Zheng Zhu , Yansong Tang

Vision-Language-Action (VLA) models are a promising path to realizing generalist embodied agents that can quickly adapt to new tasks, modalities, and environments. However, methods for interpreting and steering VLAs fall far short of…

Robotics · Computer Science 2025-09-03 Bear Häon , Kaylene Stocking , Ian Chuang , Claire Tomlin

Robotic systems are increasingly expected to operate in human-centered, unstructured environments where safety, adaptability, and generalization are essential. Vision-Language-Action (VLA) models have been proposed as a language guided…

Robotics · Computer Science 2025-10-21 Haochen Su , Cristian Meo , Francesco Stella , Andrea Peirone , Kai Junge , Josie Hughes