English

FLARE: Learning Future-Aware Latent Representations from Vision-Language Models for Autonomous Driving

Computer Vision and Pattern Recognition 2026-03-10 v2

Abstract

While Vision-Language Models (VLMs) offer rich world knowledge for end-to-end autonomous driving, current approaches heavily rely on labor-intensive language annotations (e.g., VQA) to bridge perception and control. This paradigm suffers from a fundamental mismatch between discrete linguistic tokens and continuous driving trajectories, often leading to suboptimal control policies and inefficient utilization of pre-trained knowledge. To address these challenges, we propose FLARE (Future-aware LAtent REpresentation), a novel framework that activates the visual-semantic capabilities of pre-trained VLMs without requiring language supervision. Instead of aligning with text, we introduce a self-supervised future feature prediction objective. This mechanism compels the model to anticipate scene dynamics and ego-motion directly in the latent space, enabling the learning of robust driving representations from large-scale unlabeled trajectory data. Furthermore, we integrate Group Relative Policy Optimization (GRPO) into the planning process to refine decision-making quality. Extensive experiments on the NAVSIM benchmark demonstrate that FLARE achieves state-of-the-art performance, validating the effectiveness of leveraging VLM knowledge via predictive self-supervision rather than explicit language generation.

Keywords

Cite

@article{arxiv.2601.05611,
  title  = {FLARE: Learning Future-Aware Latent Representations from Vision-Language Models for Autonomous Driving},
  author = {Chengen Xie and Chonghao Sima and Tianyu Li and Bin Sun and Junjie Wu and Zhihui Hao and Hongyang Li},
  journal= {arXiv preprint arXiv:2601.05611},
  year   = {2026}
}
R2 v1 2026-07-01T08:57:28.416Z