English

Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting

Computer Vision and Pattern Recognition 2025-01-10 v1

Abstract

Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions. It is often due to limitations like complex architectures customized for a specific dataset and inefficient multimodal handling. We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details. Additionally, our approach of reconstructing segmentlevel trajectories and lane segments from masked inputs with query drop, enables effective use of contextual information and improves generalization; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation. PerReg+ sets a new state-of-the-art performance on nuScenes [1], Argoverse 2 [2], and Waymo Open Motion Dataset (WOMD) [3]. Remarkable, our pretrained model reduces the error by 6.8% on smaller datasets, and multi-dataset training enhances generalization. In cross-domain tests, PerReg+ reduces B-FDE by 11.8% compared to its non-pretrained variant.

Keywords

Cite

@article{arxiv.2501.04815,
  title  = {Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting},
  author = {Kaouther Messaoud and Matthieu Cord and Alexandre Alahi},
  journal= {arXiv preprint arXiv:2501.04815},
  year   = {2025}
}
R2 v1 2026-06-28T21:00:29.310Z