English
Related papers

Related papers: Multi-Modal Fusion Transformer for End-to-End Auto…

200 papers

Robust semantic perception for autonomous vehicles relies on effectively combining multiple sensors with complementary strengths and weaknesses. State-of-the-art sensor fusion approaches to semantic perception often treat sensor data…

Computer Vision and Pattern Recognition · Computer Science 2026-01-27 Tim Broedermannn , Christos Sakaridis , Luigi Piccinelli , Wim Abbeloos , Luc Van Gool

Multi-modal fusion is a basic task of autonomous driving system perception, which has attracted many scholars' interest in recent years. The current multi-modal fusion methods mainly focus on camera data and LiDAR data, but pay little…

Robotics · Computer Science 2022-11-14 Yan Gong , Jianli Lu , Jiayi Wu , Wenzhuo Liu

A crucial component of an autonomous vehicle (AV) is the artificial intelligence (AI) is able to drive towards a desired destination. Today, there are different paradigms addressing the development of AI drivers. On the one hand, we find…

Computer Vision and Pattern Recognition · Computer Science 2020-10-27 Yi Xiao , Felipe Codevilla , Akhil Gurram , Onay Urfalioglu , Antonio M. López

The combination of data from multiple sensors, also known as sensor fusion or data fusion, is a key aspect in the design of autonomous robots. In particular, algorithms able to accommodate sensor fusion techniques enable increased accuracy,…

Robotics · Computer Science 2021-03-26 Li Qingqing , Jorge Peña Queralta , Tuan Nguyen Gia , Zhuo Zou , Tomi Westerlund

Current multi-modality driving frameworks normally fuse representation by utilizing attention between single-modality branches. However, the existing networks still suppress the driving performance as the Image and LiDAR branches are…

Computer Vision and Pattern Recognition · Computer Science 2024-05-14 Yiqun Duan , Xianda Guo , Zheng Zhu , Zhen Wang , Yu-Kai Wang , Chin-Teng Lin

Besides standard cameras, autonomous vehicles typically include multiple additional sensors, such as lidars and radars, which help acquire richer information for perceiving the content of the driving scene. While several recent works focus…

Computer Vision and Pattern Recognition · Computer Science 2023-08-14 Tim Broedermann , Christos Sakaridis , Dengxin Dai , Luc Van Gool

Reliable perception is essential for autonomous driving systems to operate safely under diverse real-world traffic conditions. However, camera- and LiDAR-based perception systems suffer from performance degradation under adverse weather and…

Computer Vision and Pattern Recognition · Computer Science 2026-03-17 Yue Sun , Yeqiang Qian , Zhe Wang , Tianhui Li , Chunxiang Wang , Ming Yang

Reliable detection and tracking of surrounding objects are indispensable for comprehensive motion prediction and planning of autonomous vehicles. Due to the limitations of individual sensors, the fusion of multiple sensor modalities is…

Robotics · Computer Science 2023-10-13 Phillip Karle , Felix Fent , Sebastian Huch , Florian Sauerbeck , Markus Lienkamp

The growing demand for robust scene understanding in mobile robotics and autonomous driving has highlighted the importance of integrating multiple sensing modalities. By combining data from diverse sensors like cameras and LIDARs, fusion…

Computer Vision and Pattern Recognition · Computer Science 2024-11-07 Depanshu Sani , Saket Anand

Fully autonomous driving systems require fast detection and recognition of sensitive objects in the environment. In this context, intelligent vehicles should share their sensor data with computing platforms and/or other vehicles, to detect…

Networking and Internet Architecture · Computer Science 2021-04-27 Valentina Rossi , Paolo Testolina , Marco Giordani , Michele Zorzi

There are two critical sensors for 3D perception in autonomous driving, the camera and the LiDAR. The camera provides rich semantic information such as color, texture, and the LiDAR reflects the 3D shape and locations of surrounding…

Computer Vision and Pattern Recognition · Computer Science 2022-05-31 Kaicheng Yu , Tang Tao , Hongwei Xie , Zhiwei Lin , Zhongwei Wu , Zhongyu Xia , Tingting Liang , Haiyang Sun , Jiong Deng , Dayang Hao , Yongtao Wang , Xiaodan Liang , Bing Wang

Transformer-based architectures have shown remarkable performance in vision and language tasks but pose unique challenges for safety-critical applications. This paper presents a conceptual framework for integrating Transformers into…

Software Engineering · Computer Science 2026-01-28 Sven Kirchner , Nils Purschke , Chengdong Wu , Alois Knoll

Leveraging multiple sensors is crucial for robust semantic perception in autonomous driving, as each sensor type has complementary strengths and weaknesses. However, existing sensor fusion methods often treat sensors uniformly across all…

Computer Vision and Pattern Recognition · Computer Science 2025-01-28 Tim Broedermann , Christos Sakaridis , Yuqian Fu , Luc Van Gool

With the progress of the urbanisation process, the urban transportation system is extremely critical to the development of cities and the quality of life of the citizens. Among them, it is one of the most important tasks to judge traffic…

Machine Learning · Computer Science 2023-08-17 Bodong Zhou , Jiahui Liu , Songyi Cui , Yaping Zhao

Multimodal sensor fusion is an essential capability for autonomous robots, enabling object detection and decision-making in the presence of failing or uncertain inputs. While recent fusion methods excel in normal environmental conditions,…

Computer Vision and Pattern Recognition · Computer Science 2025-08-25 Edoardo Palladin , Roland Dietze , Praveen Narayanan , Mario Bijelic , Felix Heide

Our study assesses the adversarial robustness of LiDAR-camera fusion models in 3D object detection. We introduce an attack technique that, by simply adding a limited number of physically constrained adversarial points above a car, can make…

Robotics · Computer Science 2024-01-10 Bo Yang , Xiaoyu Ji , Zizhi Jin , Yushi Cheng , Wenyuan Xu

Lidars and cameras are critical sensors that provide complementary information for 3D detection in autonomous driving. While prevalent multi-modal methods simply decorate raw lidar point clouds with camera features and feed them directly to…

Computer Vision and Pattern Recognition · Computer Science 2022-03-17 Yingwei Li , Adams Wei Yu , Tianjian Meng , Ben Caine , Jiquan Ngiam , Daiyi Peng , Junyang Shen , Bo Wu , Yifeng Lu , Denny Zhou , Quoc V. Le , Alan Yuille , Mingxing Tan

Although autonomous vehicles (AVs) are expected to revolutionize transportation, robust perception across a wide range of driving contexts remains a significant challenge. Techniques to fuse sensor data from camera, radar, and lidar sensors…

Computer Vision and Pattern Recognition · Computer Science 2022-01-19 Arnav Vaibhav Malawade , Trier Mortlock , Mohammad Abdullah Al Faruque

Predicting future trajectories of traffic agents in highly interactive environments is an essential and challenging problem for the safe operation of autonomous driving systems. On the basis of the fact that self-driving vehicles are…

Computer Vision and Pattern Recognition · Computer Science 2021-03-30 Chiho Choi , Joon Hee Choi , Jiachen Li , Srikanth Malla

Predicting future trajectories of traffic agents in highly interactive environments is an essential and challenging problem for the safe operation of autonomous driving systems. On the basis of the fact that self-driving vehicles are…

Computer Vision and Pattern Recognition · Computer Science 2021-06-15 Chiho Choi , Joon Hee Choi , Srikanth Malla , Jiachen Li