English
Related papers

Related papers: VDT-Auto: End-to-end Autonomous Driving with VLM-G…

200 papers

Large-scale Vision Language Models (LVLMs) exhibit advanced capabilities in tasks that require visual information, including object detection. These capabilities have promising applications in various industrial domains, such as autonomous…

Computer Vision and Pattern Recognition · Computer Science 2025-12-01 Haruki Sakajo , Hiroshi Takato , Hiroshi Tsutsui , Komei Soda , Hidetaka Kamigaito , Taro Watanabe

As autonomous vehicles (AVs) are increasingly deployed on public roads, understanding their real-world behaviors is critical for traffic safety analysis and regulatory oversight. However, many data-driven methods lack interpretability and…

Robotics · Computer Science 2026-03-24 Xiangyu Li , Tianyi Wang , Junfeng Jiao , Christian Claudel , Zhaomiao Guo

Traditional autonomous driving methods adopt a modular design, decomposing tasks into sub-tasks. In contrast, end-to-end autonomous driving directly outputs actions from raw sensor data, avoiding error accumulation. However, training an…

Robotics · Computer Science 2024-11-22 Zeyu Dong , Yimin Zhu , Yansong Li , Kevin Mahon , Yu Sun

Learning contextual and spatial environmental representations enhances autonomous vehicle's hazard anticipation and decision-making in complex scenarios. Recent perception systems enhance spatial understanding with sensor fusion but often…

Robotics · Computer Science 2024-01-18 Shoaib Azam , Farzeen Munir , Ville Kyrki , Moongu Jeon , Witold Pedrycz

Generating novel views of a natural scene, e.g., every-day scenes both indoors and outdoors, from a single view is an under-explored problem, even though it is an organic extension to the object-centric novel view synthesis. Existing…

Computer Vision and Pattern Recognition · Computer Science 2025-11-13 Wonbong Jang , Jonathan Tremblay , Lourdes Agapito

Zero-shot, training-free, image-based text-to-video generation is an emerging area that aims to generate videos using existing image-based diffusion models. Current methods in this space require specific architectural changes to image…

Computer Vision and Pattern Recognition · Computer Science 2025-04-10 Diljeet Jagpal , Xi Chen , Vinay P. Namboodiri

The rapid progress of multimodal large language models (MLLM) has paved the way for Vision-Language-Action (VLA) paradigms, which integrate visual perception, natural language understanding, and control within a single policy. Researchers…

In the field of autonomous driving, end-to-end deep learning models show great potential by learning driving decisions directly from sensor data. However, training these models requires large amounts of labeled data, which is time-consuming…

Computer Vision and Pattern Recognition · Computer Science 2025-03-17 Wenhao Jiang , Duo Li , Menghan Hu , Chao Ma , Ke Wang , Zhipeng Zhang

The core challenge in automotive exterior design is balancing subjective aesthetics with objective aerodynamic performance while dramatically accelerating the development cycle. To address this, we propose a novel, LLM-driven multi-agent…

Computational Engineering, Finance, and Science · Computer Science 2025-08-06 Xinyu Jin , Shengmao Yan , Qingtao Wang , Shisong Deng , Yanzhen Jiang , Shuangyao Zhao

End-to-end learning has emerged as a transformative paradigm in autonomous driving. However, the inherently multimodal nature of driving behaviors and the generalization challenges in long-tail scenarios remain critical obstacles to robust…

Robotics · Computer Science 2025-05-27 Rui Zhao , Yuze Fan , Ziguo Chen , Fei Gao , Zhenhai Gao

Autonomous driving presents a complex challenge, which is usually addressed with artificial intelligence models that are end-to-end or modular in nature. Within the landscape of modular approaches, a bio-inspired neural circuit policy model…

Computer Vision and Pattern Recognition · Computer Science 2024-04-03 Anass Bairouk , Mirjana Maras , Simon Herlin , Alexander Amini , Marc Blanchon , Ramin Hasani , Patrick Chareyre , Daniela Rus

Recently, deep-learning based approaches have achieved impressive performance for autonomous driving. However, end-to-end vision-based methods typically have limited interpretability, making the behaviors of the deep networks difficult to…

Computer Vision and Pattern Recognition · Computer Science 2021-04-28 Hengli Wang , Peide Cai , Yuxiang Sun , Lujia Wang , Ming Liu

The evolution of autonomous driving towards full automation demands robust interactive capabilities; however, the development of Vision-Language-Action (VLA) models is constrained by the sparsity of interactive scenarios and inadequate…

Computer Vision and Pattern Recognition · Computer Science 2026-02-25 Haojie Feng , Peizhi Zhang , Mengjie Tian , Xinrui Zhang , Zhuoren Li , Junpeng Huang , Xiurong Wang , Junfan Zhu , Jianzhou Wang , Dongxiao Yin , Lu Xiong

Integrating large language models (LLMs) into autonomous driving motion planning has recently emerged as a promising direction, offering enhanced interpretability, better controllability, and improved generalization in rare and long-tail…

Artificial Intelligence · Computer Science 2025-07-29 Zhipeng Tang , Sha Zhang , Jiajun Deng , Chenjie Wang , Guoliang You , Yuting Huang , Xinrui Lin , Yanyong Zhang

Recent advances in end-to-end autonomous driving leverage multi-view images to construct BEV representations for motion planning. In motion planning, autonomous vehicles need considering both hard constraints imposed by geometrically…

Vision-Language-Action (VLA) models have recently demonstrated strong performance across embodied tasks. Modern VLAs commonly employ diffusion action experts to efficiently generate high-precision continuous action chunks, while…

Computer Vision and Pattern Recognition · Computer Science 2026-03-20 Chen Zhao , Zhuoran Wang , Haoyang Li , Shifeng Bao , Guanlin Li , Youhe Feng , Yang Li , Jie Tang , Jing Zhang

Vision-Language Models (VLMs) have demonstrated notable promise in autonomous driving by offering the potential for multimodal reasoning through pretraining on extensive image-text pairs. However, adapting these models from broad web-scale…

Robotics · Computer Science 2025-06-18 Yupeng Zhou , Can Cui , Juntong Peng , Zichong Yang , Juanwu Lu , Jitesh H Panchal , Bin Yao , Ziran Wang

Diffusion Transformers (DiT) trained with flow matching in a VAE latent space have unified visual generation across images and videos. A natural next step toward a single architecture for both generation (visual synthesis) and understanding…

Computation and Language · Computer Science 2026-05-11 Jiaxiu Jiang , Jingjing Ren , Wenbo Li , Bo Wang , Haoze Sun , Yijun Yang , Jianhui Liu , Yanbing Zhang , Shenghe Zheng , Yuan Zhang , Haoyang Huang , Nan Duan , Wangmeng Zuo

Vision-Language Models (VLMs) are increasingly deployed in real-time applications such as autonomous driving and human-computer interaction, which demand fast and reliable responses based on accurate perception. To meet these requirements,…

Computer Vision and Pattern Recognition · Computer Science 2025-11-18 Chen Qian , Xinran Yu , Zewen Huang , Danyang Li , Qiang Ma , Fan Dang , Xuan Ding , Guangyong Shang , Zheng Yang

BEV perception is of great importance in the field of autonomous driving, serving as the cornerstone of planning, controlling, and motion prediction. The quality of the BEV feature highly affects the performance of BEV perception. However,…

Computer Vision and Pattern Recognition · Computer Science 2023-03-16 Jiayu Zou , Zheng Zhu , Yun Ye , Xingang Wang