English
Related papers

Related papers: XLM for Autonomous Driving Systems: A Comprehensiv…

200 papers

The rapid development of Artificial Intelligence (AI) has revolutionized numerous fields, with large language models (LLMs) and computer vision (CV) systems driving advancements in natural language understanding and visual processing,…

Computation and Language · Computer Science 2024-12-04 Yunkai Dang , Kaichen Huang , Jiahao Huo , Yibo Yan , Sirui Huang , Dongrui Liu , Mengxi Gao , Jie Zhang , Chen Qian , Kun Wang , Yong Liu , Jing Shao , Hui Xiong , Xuming Hu

Connecting text and visual modalities plays an essential role in generative intelligence. For this reason, inspired by the success of large language models, significant research efforts are being devoted to the development of Multimodal…

Computer Vision and Pattern Recognition · Computer Science 2024-06-07 Davide Caffagni , Federico Cocchi , Luca Barsellotti , Nicholas Moratelli , Sara Sarto , Lorenzo Baraldi , Lorenzo Baraldi , Marcella Cornia , Rita Cucchiara

Language models (LMs) are machine learning models designed to predict linguistic patterns by estimating the probability of word sequences based on large-scale datasets, such as text. LMs have a wide range of applications in natural language…

This paper provides a comprehensive survey of the latest research on multilingual large language models (MLLMs). MLLMs not only are able to understand and generate language across linguistic boundaries, but also represent an important…

Computation and Language · Computer Science 2024-11-20 Shaolin Zhu , Supryadi , Shaoyang Xu , Haoran Sun , Leiyu Pan , Menglong Cui , Jiangcun Du , Renren Jin , António Branco , Deyi Xiong

Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contributions in this direction. These works…

Computation and Language · Computer Science 2024-10-18 Humza Naveed , Asad Ullah Khan , Shi Qiu , Muhammad Saqib , Saeed Anwar , Muhammad Usman , Naveed Akhtar , Nick Barnes , Ajmal Mian

Large language models (LLMs) can handle a wide variety of general tasks with simple prompts, without the need for task-specific training. Multimodal Large Language Models (MLLMs), built upon LLMs, have demonstrated impressive potential in…

Computer Vision and Pattern Recognition · Computer Science 2025-03-25 Tao Yu , Yi-Fan Zhang , Chaoyou Fu , Junkang Wu , Jinda Lu , Kun Wang , Xingyu Lu , Yunhang Shen , Guibin Zhang , Dingjie Song , Yibo Yan , Tianlong Xu , Qingsong Wen , Zhang Zhang , Yan Huang , Liang Wang , Tieniu Tan

The rapid rise of Large Language Models (LLMs) is transforming traffic and transportation research, with significant advancements emerging between the years 2023 and 2025 -- a period marked by the inception and swift growth of adopting and…

Computational Engineering, Finance, and Science · Computer Science 2025-03-28 Yimo Yan , Yejia Liao , Guanhao Xu , Ruili Yao , Huiying Fan , Jingran Sun , Xia Wang , Jonathan Sprinkle , Ziyan An , Meiyi Ma , Xi Cheng , Tong Liu , Zemian Ke , Bo Zou , Matthew Barth , Yong-Hong Kuo

We provide a sober look at the application of Multimodal Large Language Models (MLLMs) in autonomous driving, challenging common assumptions about their ability to interpret dynamic driving scenarios. Despite advances in models like GPT-4o,…

Robotics · Computer Science 2024-10-29 Shiva Sreeram , Tsun-Hsuan Wang , Alaa Maalouf , Guy Rosman , Sertac Karaman , Daniela Rus

In the field of autonomous surface vehicles (ASVs), devising decision-making and obstacle avoidance solutions that address maritime COLREGs (Collision Regulations), primarily defined for human operators, has long been a pressing challenge.…

Robotics · Computer Science 2025-04-09 Klinsmann Agyei , Pouria Sarhadi , Wasif Naeem

This paper proposes a novel Large Vision-Language Model (LVLM) and Model Predictive Control (MPC) integration framework that delivers both task scalability and safety for Autonomous Driving (AD). LVLMs excel at high-level task planning…

Robotics · Computer Science 2025-07-16 Kazuki Atsuta , Kohei Honda , Hiroyuki Okuda , Tatsuya Suzuki

Large Language Models (LLMs), with their abilities in knowledge acquisition and reasoning, can potentially enhance the various aspects of Self-adaptive Systems (SAS). Yet, the potential of LLMs in SAS remains largely unexplored and…

Software Engineering · Computer Science 2024-01-17 Jialong Li , Mingyue Zhang , Nianyu Li , Danny Weyns , Zhi Jin , Kenji Tei

Multimodal large language models (MLLMs) hold the potential to enhance autonomous driving by combining domain-independent world knowledge with context-specific language guidance. Their integration into autonomous driving systems shows…

Computer Vision and Pattern Recognition · Computer Science 2025-03-17 Tin Stribor Sohn , Philipp Reis , Maximilian Dillitzer , Johannes Bach , Jason J. Corso , Eric Sax

Autonomous driving is a complex task which requires advanced decision making and control algorithms. Understanding the rationale behind the autonomous vehicles' decision is crucial to ensure their safe and effective operation on highway…

Robotics · Computer Science 2024-05-24 Mustafa Yildirim , Barkin Dagda , Saber Fallah

Large Vision Language Models (LVLMs) have shown strong capabilities in understanding and analyzing visual scenes across various domains. However, in the context of autonomous driving, their limited comprehension of 3D environments restricts…

Computer Vision and Pattern Recognition · Computer Science 2025-05-02 Jannik Lübberstedt , Esteban Rivera , Nico Uhlemann , Markus Lienkamp

The advent of Large Multimodal Models (LMMs) offers a promising technology to tackle the limitations of modular design in autonomous driving, which often falters in open-world scenarios requiring sustained environmental understanding and…

Robotics · Computer Science 2026-01-21 Long Zhang , Yuchen Xia , Bingqing Wei , Zhen Liu , Shiwen Mao , Zhu Han , Mohsen Guizani

Motion prediction is among the most fundamental tasks in autonomous driving. Traditional methods of motion forecasting primarily encode vector information of maps and historical trajectory data of traffic participants, lacking a…

Computer Vision and Pattern Recognition · Computer Science 2024-07-31 Xiaoji Zheng , Lixiu Wu , Zhijie Yan , Yuanrong Tang , Hao Zhao , Chen Zhong , Bokui Chen , Jiangtao Gong

Deep reinforcement learning (DRL) shows promising potential for autonomous driving decision-making. However, DRL demands extensive computational resources to achieve a qualified policy in complex driving scenarios due to its low learning…

Robotics · Computer Science 2024-12-25 Hao Pang , Zhenpo Wang , Guoqiang Li

Mobility analysis is a crucial element in the research area of transportation systems. Forecasting traffic information offers a viable solution to address the conflict between increasing transportation demands and the limitations of…

Machine Learning · Computer Science 2025-02-24 Zijian Zhang , Yujie Sun , Zepu Wang , Yuqi Nie , Xiaobo Ma , Ruolin Li , Peng Sun , Xuegang Ban

In recent years, the integration of large language models (LLMs) has revolutionized the field of robotics, enabling robots to communicate, understand, and reason with human-like proficiency. This paper explores the multifaceted impact of…

Robotics · Computer Science 2024-08-16 Yeseung Kim , Dohyun Kim , Jieun Choi , Jisang Park , Nayoung Oh , Daehyung Park

Building effective machine learning (ML) workflows to address complex tasks is a primary focus of the Automatic ML (AutoML) community and a critical step toward achieving artificial general intelligence (AGI). Recently, the integration of…

Machine Learning · Computer Science 2024-12-30 Yang Gu , Hengyu You , Jian Cao , Muran Yu , Haoran Fan , Shiyou Qian