English
Related papers

Related papers: LMDrive: Closed-Loop End-to-End Driving with Large…

200 papers

The evolution of autonomous driving has made remarkable advancements in recent years, evolving into a tangible reality. However, a human-centric large-scale adoption hinges on meeting a variety of multifaceted requirements. To ensure that…

Computation and Language · Computer Science 2023-12-21 Yi Yang , Qingwen Zhang , Ci Li , Daniel Simões Marta , Nazre Batool , John Folkesson

Artificial intelligence (AI) plays a crucial role in autonomous driving (AD) research, propelling its development towards intelligence and efficiency. Currently, the development of AD technology follows two main technical paths:…

Artificial Intelligence · Computer Science 2024-12-03 Yuxuan Zhu , Shiyi Wang , Wenqing Zhong , Nianchen Shen , Yunqi Li , Siqi Wang , Zhiheng Li , Cathy Wu , Zhengbing He , Li Li

We provide a sober look at the application of Multimodal Large Language Models (MLLMs) in autonomous driving, challenging common assumptions about their ability to interpret dynamic driving scenarios. Despite advances in models like GPT-4o,…

Robotics · Computer Science 2024-10-29 Shiva Sreeram , Tsun-Hsuan Wang , Alaa Maalouf , Guy Rosman , Sertac Karaman , Daniela Rus

Although planning is a crucial component of the autonomous driving stack, researchers have yet to develop robust planning algorithms that are capable of safely handling the diverse range of possible driving scenarios. Learning-based…

Artificial Intelligence · Computer Science 2024-01-02 S P Sharan , Francesco Pittaluga , Vijay Kumar B G , Manmohan Chandraker

Recent advancements in autonomous vehicles (AVs) use Large Language Models (LLMs) to perform well in normal driving scenarios. However, ensuring safety in dynamic, high-risk environments and managing safety-critical long-tail events remain…

Artificial Intelligence · Computer Science 2024-12-20 Zhiyuan Zhou , Heye Huang , Boqi Li , Shiyue Zhao , Yao Mu , Jianqiang Wang

The rapid rise of Large Language Models (LLMs) is transforming traffic and transportation research, with significant advancements emerging between the years 2023 and 2025 -- a period marked by the inception and swift growth of adopting and…

Computational Engineering, Finance, and Science · Computer Science 2025-03-28 Yimo Yan , Yejia Liao , Guanhao Xu , Ruili Yao , Huiying Fan , Jingran Sun , Xia Wang , Jonathan Sprinkle , Ziyan An , Meiyi Ma , Xi Cheng , Tong Liu , Zemian Ke , Bo Zou , Matthew Barth , Yong-Hong Kuo

Deep learning architectures with powerful reasoning capabilities have driven significant advancements in autonomous driving technology. Large language models (LLMs) applied in this field can describe driving scenes and behaviors with a…

Artificial Intelligence · Computer Science 2024-10-01 Yizhou Huang , Yihua Cheng , Kezhi Wang

In recent years, the integration of large language models (LLMs) has revolutionized the field of robotics, enabling robots to communicate, understand, and reason with human-like proficiency. This paper explores the multifaceted impact of…

Robotics · Computer Science 2024-08-16 Yeseung Kim , Dohyun Kim , Jieun Choi , Jisang Park , Nayoung Oh , Daehyung Park

The human ability to learn, generalize, and control complex manipulation tasks through multi-modality feedback suggests a unique capability, which we refer to as dexterity intelligence. Understanding and assessing this intelligence is a…

Robotics · Computer Science 2025-12-03 Fanlong Zeng , Wensheng Gan , Zezheng Huai , Lichao Sun , Hechang Chen , Yongheng Wang , Ning Liu , Philip S. Yu

Trajectory prediction serves as a critical functionality in autonomous driving, enabling the anticipation of future motion paths for traffic participants such as vehicles and pedestrians, which is essential for driving safety. Although…

Robotics · Computer Science 2025-09-16 Wei Dai , Shengen Wu , Wei Wu , Zhenhao Wang , Sisuo Lyu , Haicheng Liao , Limin Yu , Weiping Ding , Runwei Guan , Yutao Yue

Pre-trained large language models (LLMs) have demonstrated strong common-sense reasoning abilities, making them promising for robotic navigation and planning tasks. However, despite recent progress, bridging the gap between language…

Robotics · Computer Science 2025-12-29 Mingfeng Yuan , Letian Wang , Steven L. Waslander

Integrating large language models (LLMs) into autonomous driving has attracted significant attention with the hope of improving generalization and explainability. However, existing methods often focus on either driving or vision-language…

Computer Vision and Pattern Recognition · Computer Science 2025-03-13 Katrin Renz , Long Chen , Elahe Arani , Oleg Sinavski

Autonomous Driving Systems (ADSs) are revolutionizing transportation by reducing human intervention, improving operational efficiency, and enhancing safety. Large Language Models (LLMs) have been integrated into ADSs to support high-level…

Multiagent Systems · Computer Science 2025-10-15 Yaozu Wu , Dongyuan Li , Yankai Chen , Renhe Jiang , Henry Peng Zou , Wei-Chieh Huang , Yangning Li , Liancheng Fang , Zhen Wang , Philip S. Yu

Large-scale Vision Language Models (LVLMs) exhibit advanced capabilities in tasks that require visual information, including object detection. These capabilities have promising applications in various industrial domains, such as autonomous…

Computer Vision and Pattern Recognition · Computer Science 2025-12-01 Haruki Sakajo , Hiroshi Takato , Hiroshi Tsutsui , Komei Soda , Hidetaka Kamigaito , Taro Watanabe

Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the widespread deployment of autonomous vehicles (AVs). In this paper, we present LLaDA, a simple yet powerful…

Robotics · Computer Science 2024-04-12 Boyi Li , Yue Wang , Jiageng Mao , Boris Ivanovic , Sushant Veer , Karen Leung , Marco Pavone

Large Language Models (LLMs) have garnered significant attention for their ability to understand text and images, generate human-like text, and perform complex reasoning tasks. However, their ability to generalize this advanced reasoning…

Computer Vision and Pattern Recognition · Computer Science 2025-02-04 Mehdi Azarafza , Mojtaba Nayyeri , Charles Steinmetz , Steffen Staab , Achim Rettberg

Deep reinforcement learning (DRL) shows promising potential for autonomous driving decision-making. However, DRL demands extensive computational resources to achieve a qualified policy in complex driving scenarios due to its low learning…

Robotics · Computer Science 2024-12-25 Hao Pang , Zhenpo Wang , Guoqiang Li

Recent advancements in foundation models (FMs) have unlocked new prospects in autonomous driving, yet the experimental settings of these studies are preliminary, over-simplified, and fail to capture the complexity of real-world driving…

Computer Vision and Pattern Recognition · Computer Science 2024-10-16 Yidong Huang , Jacob Sansom , Ziqiao Ma , Felix Gervits , Joyce Chai

The utilization of Large Language Models (LLMs) within the realm of reinforcement learning, particularly as planners, has garnered a significant degree of attention in recent scholarly literature. However, a substantial proportion of…

Robotics · Computer Science 2024-07-30 Yiqun Duan , Qiang Zhang , Renjing Xu

Autonomous driving is a complex task which requires advanced decision making and control algorithms. Understanding the rationale behind the autonomous vehicles' decision is crucial to ensure their safe and effective operation on highway…

Robotics · Computer Science 2024-05-24 Mustafa Yildirim , Barkin Dagda , Saber Fallah