English
Related papers

Related papers: Information-Theoretic Constraints for Continual Vi…

200 papers

Utilizing large language models (LLMs) to compose off-the-shelf visual tools represents a promising avenue of research for developing robust visual assistants capable of addressing diverse visual tasks. However, these methods often overlook…

Computer Vision and Pattern Recognition · Computer Science 2024-04-11 Zhi Gao , Yuntao Du , Xintong Zhang , Xiaojian Ma , Wenjuan Han , Song-Chun Zhu , Qing Li

We present ProgVLA, a compact vision-language-action (VLA) model designed for reliable robot manipulation under tight compute and memory budgets. The model specifically focuses on efficiently processing long multi-modal sequences by…

Robotics · Computer Science 2026-05-28 Seungsu Kim , Jinyoung Choi , Seungmin Baek , Jean-Michel Renders

Current vision-language models (VLMs) still exhibit inferior performance on knowledge-intensive tasks, primarily due to the challenge of accurately encoding all the associations between visual objects and scenes to their corresponding…

Computation and Language · Computer Science 2024-10-16 Jingyuan Qi , Zhiyang Xu , Rulin Shao , Yang Chen , Jin Di , Yu Cheng , Qifan Wang , Lifu Huang

We present a novel visual instruction tuning strategy to improve the zero-shot task generalization of multimodal large language models by building a firm text-only knowledge base. Existing work lacks sufficient experimentation on the…

Computation and Language · Computer Science 2025-07-01 Jianhong Tu , Zhuohao Ni , Nicholas Crispino , Zihao Yu , Michael Bendersky , Beliz Gunel , Ruoxi Jia , Xin Liu , Lingjuan Lyu , Dawn Song , Chenguang Wang

Class-incremental learning requires a learning system to continually learn knowledge of new classes and meanwhile try to preserve previously learned knowledge of old classes. As current state-of-the-art methods based on Vision-Language…

Computer Vision and Pattern Recognition · Computer Science 2025-12-11 Jiantao Tan , Peixian Ma , Tong Yu , Wentao Zhang , Ruixuan Wang

In this paper, we introduce a novel kinematics-rich vision-language-action (VLA) task, in which language commands densely encode diverse kinematic attributes (such as direction, trajectory, orientation, and relative displacement) from…

Robotics · Computer Science 2026-03-19 Gaoge Han , Zhengqing Gao , Ziwen Li , Jiaxin Huang , Shaoli Huang , Fakhri Karray , Mingming Gong , Tongliang Liu

Vision-Language-Action (VLA) models have recently emerged as powerful general-purpose policies for robotic manipulation, benefiting from large-scale multi-modal pre-training. However, they often fail to generalize reliably in…

Robotics · Computer Science 2025-12-02 Hongyin Zhang , Shuo Zhang , Junxi Jin , Qixin Zeng , Runze Li , Donglin Wang

Recent reasoning-augmented Vision-Language-Action (VLA) models have improved the interpretability of end-to-end autonomous driving by generating intermediate reasoning traces. Yet these models primarily describe what they perceive and…

Vision-Language-Action (VLA) models typically map visual observations and linguistic instructions directly to control signals. This "black-box" mapping forces a single forward pass to simultaneously handle instruction interpretation,…

Robotics · Computer Science 2026-05-12 Zixuan Wang , Yuxin Chen , Yuqi Liu , Jinhui Ye , Pengguang Chen , Changsheng Lu , Shu Liu , Bei Yu , Jiaya Jia

Vision-Language-Action (VLA) models are a promising path toward embodied intelligence, yet they often overlook the predictive and temporal-causal structure underlying visual dynamics. World-model VLAs address this by predicting future…

Computer Vision and Pattern Recognition · Computer Science 2026-03-04 Fuxiang Yang , Donglin Di , Lulu Tang , Xuancheng Zhang , Lei Fan , Hao Li , Chen Wei , Tonghua Su , Baorui Ma

Vision-Language-Action (VLA) models offer a compelling framework for tackling complex robotic manipulation tasks, but they are often expensive to train. In this paper, we propose a novel VLA approach that leverages the competitive…

Robotics · Computer Science 2025-12-23 Max Argus , Jelena Bratulic , Houman Masnavi , Maxim Velikanov , Nick Heppert , Abhinav Valada , Thomas Brox

Vision-language-action (VLA) models have emerged as the next generation of models in robotics. However, despite leveraging powerful pre-trained Vision-Language Models (VLMs), existing end-to-end VLA systems often lose key capabilities…

Robotics · Computer Science 2025-06-02 Zhongyi Zhou , Yichen Zhu , Junjie Wen , Chaomin Shen , Yi Xu

While Vision-Language-Action (VLA) models have revolutionized autonomous driving by unifying perception and planning, their reliance on explicit textual Chain-of-Thought (CoT) leads to semantic-perceptual decoupling and perceptual-symbolic…

Computer Vision and Pattern Recognition · Computer Science 2026-03-13 Yuechen Luo , Fang Li , Shaoqing Xu , Yang Ji , Zehan Zhang , Bing Wang , Yuannan Shen , Jianwei Cui , Long Chen , Guang Chen , Hangjun Ye , Zhi-Xin Yang , Fuxi Wen

Contrastive learning has achieved remarkable success in self-supervised representation learning, often guided by information-theoretic objectives such as mutual information maximization. Motivated by the limitations of static augmentations…

Machine Learning · Computer Science 2026-03-16 Jiansong Zhang , Zhuoqin Yang , Xu Wu , Xiaoling Luo , Peizhong Liu , Linlin Shen

Vision Language Action (VLA) models have recently shown great potential in bridging multimodal perception with robotic control. However, existing methods often rely on direct fine-tuning of pre-trained Vision-Language Models (VLMs), feeding…

Robotics · Computer Science 2026-02-04 Kun Wang , Xiao Feng , Mingcheng Qu , Tonghua Su

The stability-plasticity dilemma is a major challenge in continual learning, as it involves balancing the conflicting objectives of maintaining performance on previous tasks while learning new tasks. In this paper, we propose the…

Machine Learning · Computer Science 2024-03-06 Haneol Kang , Dong-Wan Choi

The Vision-Language-Action (VLA) models have demonstrated remarkable performance on embodied tasks and shown promising potential for real-world applications. However, current VLAs still struggle to produce consistent and precise…

Computer Vision and Pattern Recognition · Computer Science 2025-12-09 Ziwen Li , Xin Wang , Hanlue Zhang , Runnan Chen , Runqi Lin , Xiao He , Han Huang , Yandong Guo , Fakhri Karray , Tongliang Liu , Mingming Gong

Continual learning enables pre-trained generative vision-language models (VLMs) to incorporate knowledge from new tasks without retraining data from previous ones. Recent methods update a visual projector to translate visual information for…

Computer Vision and Pattern Recognition · Computer Science 2025-08-04 Hyundong Jin , Hyung Jin Chang , Eunwoo Kim

Multi-task ``vision-language-action'' (VLA) models have recently demonstrated increasing promise as generalist foundation models for robotics, achieving non-trivial performance out of the box on new tasks in new environments. However, for…

Robotics · Computer Science 2025-08-05 Kaustubh Sridhar , Souradeep Dutta , Dinesh Jayaraman , Insup Lee

Visual-Language-Action (VLA) models report impressive success rates on robotic manipulation benchmarks, yet these results may mask fundamental weaknesses in robustness. We perform a systematic vulnerability analysis by introducing…