Classical autonomous driving systems connect perception and prediction modules via hand-crafted bounding-box interfaces, limiting information flow and propagating errors to downstream tasks. Recent research aims to develop end-to-end models that jointly address perception and prediction; however, they often fail to fully exploit the synergy between appearance and motion cues, relying mainly on short-term visual features. We follow the idea of "looking backward to look forward", and propose MASAR, a novel fully differentiable framework for joint 3D detection and trajectory forecasting compatible with any transformer-based 3D detector. MASAR employs an object-centric spatio-temporal mechanism that jointly encodes appearance and motion features. By predicting past trajectories and refining them using guidance from appearance cues, MASAR captures long-term temporal dependencies that enhance future trajectory forecasting. Experiments conducted on the nuScenes dataset demonstrate MASAR's effectiveness, showing improvements of over 20% in minADE and minFDE while maintaining robust detection performance. Code and models are available at https://github.com/aminmed/MASAR.
@article{arxiv.2602.13003,
title = {MASAR: Motion-Appearance Synergy Refinement for Joint Detection and Trajectory Forecasting},
author = {Mohammed Amine Bencheikh Lehocine and Julian Schmidt and Frank Moosmann and Dikshant Gupta and Fabian Flohr},
journal= {arXiv preprint arXiv:2602.13003},
year = {2026}
}
Comments
Accepted to the 2026 IEEE International Conference on Robotics and Automation (ICRA 2026)