Recent advances in foundation models (FMs), including large language models (LLMs), vision-language models (VLMs), and world models, have opened new opportunities for autonomous driving systems (ADSs) in perception, reasoning, decision-making, and interaction. However, ADSs are safety-critical cyber-physical systems, and integrating FMs into them raises substantial software engineering challenges in data curation, system design, deployment, evaluation, and assurance. To clarify this rapidly evolving landscape, we present an initial roadmap, grounded in a structured literature review, for integrating FMs into autonomous driving across three dimensions: FM infrastructure, in-vehicle integration, and practical deployment. For each dimension, we summarize the state of the art, identify key challenges, and highlight open research opportunities. Based on this analysis, we outline research directions for building reliable, safe, and trustworthy FM-enabled ADSs.
@article{arxiv.2504.00911,
title = {Foundation Models for Autonomous Driving System: An Initial Roadmap},
author = {Xiongfei Wu and Mingfei Cheng and Xiaoning Ren and Qiang Hu and Jianlang Chen and Yuheng Huang and Maxime Cordy and Yao Zhang and Xiaofei Xie and Lei Ma and Yves Le Traon},
journal= {arXiv preprint arXiv:2504.00911},
year = {2026}
}
Comments
To appear in ACM Transactions on Software Engineering and Methodology (TOSEM)