中文
相关论文

相关论文: One Token Per Frame: Reconsidering Visual Bandwidt…

200 篇论文

Multimodal instruction tuning is the de facto recipe for adapting vision language models (VLMs), yet instruction data are highly redundant, making data selection critical for training efficiency. Existing methods derive selection signals…

计算机视觉与模式识别 · 计算机科学 2026-05-27 Mingkang Dong , Hongyi Cai , Xiwen Lei , Jie Li , Tao Zhang , Muxin Pu

Vision-Language-Action (VLA) models have recently demonstrated strong performance across embodied tasks. Modern VLAs commonly employ diffusion action experts to efficiently generate high-precision continuous action chunks, while…

计算机视觉与模式识别 · 计算机科学 2026-03-20 Chen Zhao , Zhuoran Wang , Haoyang Li , Shifeng Bao , Guanlin Li , Youhe Feng , Yang Li , Jie Tang , Jing Zhang

Vision-Language-Action (VLA) models unify perception, reasoning, and control within a single policy, yet their multi-billion-parameter backbones and diffusion-based action heads make on-device deployment prohibitively expensive. Prior…

计算机视觉与模式识别 · 计算机科学 2026-05-28 Xinyu Wang , Mingze Li , Sicheng Lyu , Dongxiu Liu , Kaicheng Yang , Ziyu Zhao , Yufei Cui , Xiao-Wen Chang , Peng Lu

Vision-language models (VLMs) have achieved remarkable multimodal understanding and reasoning capabilities, yet remain computationally expensive due to dense visual tokenization. Existing efficiency approaches either merge redundant visual…

计算机视觉与模式识别 · 计算机科学 2026-03-30 Aditya Kumar Singh , Hitesh Kandala , Pratik Prabhanjan Brahma , Zicheng Liu , Emad Barsoum

Unified models aim to support both understanding and generation by encoding images into discrete tokens and processing them alongside text within a single autoregressive framework. This unified design offers architectural simplicity and…

计算机视觉与模式识别 · 计算机科学 2026-03-13 Ziyao Wang , Chen Chen , Jingtao Li , Weiming Zhuang , Jiabo Huang , Ang Li , Lingjuan Lyu

General-purpose robots must master long-horizon manipulation, defined as tasks involving multiple kinematic structure changes (e.g., attaching or detaching objects) in unstructured environments. While Vision-Language-Action (VLA) models…

机器人学 · 计算机科学 2026-02-26 Yue Yang , Shuo Cheng , Yu Fang , Homanga Bharadhwaj , Mingyu Ding , Gedas Bertasius , Daniel Szafir

Deploying powerful Vision-Language-Action (VLA) models on edge devices is limited by their massive size. In this paper, we take a deployment-oriented view of VLA training: we target efficiency through model design and optimization, rather…

机器人学 · 计算机科学 2026-03-03 Hongyu Wang , Chuyan Xiong , Ruiping Wang , Xilin Chen

Vision-Language-Action (VLA) models enable instruction-following embodied control, but their large compute and memory footprints hinder deployment on resource-constrained robots and edge platforms. While reducing weights to 1-bit precision…

机器学习 · 计算机科学 2026-02-17 Xin Yan , Zhenglin Wan , Feiyang Ye , Xingrui Yu , Hangyu Du , Yang You , Ivor Tsang

Vision-Language-Action (VLA) models adapt large vision-language backbones to map images and instructions into robot actions. However, prevailing VLAs either generate actions auto-regressively in a fixed left-to-right order or attach…

计算机视觉与模式识别 · 计算机科学 2025-12-23 Zhixuan Liang , Yizhuo Li , Tianshuo Yang , Chengyue Wu , Sitong Mao , Tian Nian , Liuao Pei , Shunbo Zhou , Xiaokang Yang , Jiangmiao Pang , Yao Mu , Ping Luo

Prevalent Vision-Language-Action (VLA) models are typically built upon Multimodal Large Language Models (MLLMs) and demonstrate exceptional proficiency in semantic understanding, but they inherently lack the capability to deduce physical…

Building generalist embodied agents requires integrating perception, language understanding, and action, which are core capabilities addressed by Vision-Language-Action (VLA) approaches based on multimodal foundation models, including…

机器人学 · 计算机科学 2026-04-08 StarVLA Community

Vision-Language Models (VLMs) have become essential backbones of modern multimodal intelligence, yet their outputs remain prone to hallucination-plausible text misaligned with visual inputs. Existing alignment approaches often rely on…

计算机视觉与模式识别 · 计算机科学 2025-10-28 Kejia Chen , Jiawen Zhang , Jiacong Hu , Kewei Gao , Jian Lou , Zunlei Feng , Mingli Song

Executing language-conditioned tasks in dynamic visual environments remains a central challenge in embodied AI. Existing Vision-Language-Action (VLA) models predominantly adopt reactive state-to-action mappings, often leading to…

机器人学 · 计算机科学 2025-09-10 Qi Lv , Weijie Kong , Hao Li , Jia Zeng , Zherui Qiu , Delin Qu , Haoming Song , Qizhi Chen , Xiang Deng , Jiangmiao Pang

Fine-tuning vision-language models (VLMs) on robot teleoperation data to create vision-language-action (VLA) models is a promising paradigm for training generalist policies, but it suffers from a fundamental tradeoff: learning to produce…

机器人学 · 计算机科学 2025-09-29 Asher J. Hancock , Xindi Wu , Lihan Zha , Olga Russakovsky , Anirudha Majumdar

The Visual-Language-Action (VLA) models can follow text instructions according to visual observations of the surrounding environment. This ability to map multimodal inputs to actions is derived from the training of the VLA model on…

计算机视觉与模式识别 · 计算机科学 2025-09-24 Jinyue Bian , Zhaoxing Zhang , Zhengyu Liang , Shiwei Zheng , Shengtao Zhang , Rong Shen , Chen Yang , Anzhou Hou

The Large Vision-Language Model (LVLM) has enhanced the performance of various downstream tasks in visual-language understanding. Most existing approaches encode images and videos into separate feature spaces, which are then fed as inputs…

计算机视觉与模式识别 · 计算机科学 2024-10-02 Bin Lin , Yang Ye , Bin Zhu , Jiaxi Cui , Munan Ning , Peng Jin , Li Yuan

Autonomous driving requires reasoning about how the environment evolves and planning actions accordingly. Existing world-model-based approaches typically predict future scenes first and plan afterwards, resulting in open-loop imagination…

机器人学 · 计算机科学 2026-03-31 Qiqi Liu , Huan Xu , Jingyu Li , Bin Sun , Zhihui Hao , Dangen She , Xiatian Zhu , Li Zhang

Amidst the advancements in image-based Large Vision-Language Models (image-LVLM), the transition to video-based models (video-LVLM) is hindered by the limited availability of quality video data. This paper addresses the challenge by…

计算机视觉与模式识别 · 计算机科学 2024-06-13 Shimin Chen , Yitian Yuan , Shaoxiang Chen , Zequn Jie , Lin Ma

Despite remarkable progress in Vision--Language--Action (VLA) models, a central bottleneck remains underexamined: the data infrastructure that underlies embodied learning. In this survey, we argue that future advances in VLA will depend…

机器人学 · 计算机科学 2026-04-28 Ziyao Wang , Bingying Wang , Hanrong Zhang , Tingting Du , Tianyang Chen , Guoheng Sun , Yexiao He , Zheyu Shen , Wanghao Ye , Ang Li

Existing Vision-Language-Action (VLA) models often suffer from feature collapse and low training efficiency because they entangle high-level perception with sparse, embodiment-specific action supervision. Since these models typically rely…

计算机视觉与模式识别 · 计算机科学 2026-05-19 Haitao Lin , Hanyang Yu , Jingshun Huang , He Zhang , Yonggen Ling , Ping Tan , Xiangyang Xue , Yanwei Fu