Accurately predicting human behaviors is crucial for mobile robots operating in human-populated environments. While prior research primarily focuses on predicting actions in single-human scenarios from an egocentric view, several robotic applications require understanding multiple human behaviors from a third-person perspective. To this end, we present CAMP-VLM (Context-Aware Multi-human behavior Prediction): a Vision Language Model (VLM)-based framework that incorporates contextual features from visual input and spatial awareness from scene graphs to enhance prediction of humans-scene interactions. Due to the lack of suitable datasets for multi-human behavior prediction from an observer view, we perform fine-tuning of CAMP-VLM with synthetic human behavior data generated by a photorealistic simulator, and evaluate the resulting models on both synthetic and real-world sequences to assess their generalization capabilities. Leveraging Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), CAMP-VLM outperforms the best-performing baseline by up to 66.9% in prediction accuracy.
@article{arxiv.2512.15957,
title = {Seeing is Believing (and Predicting): Context-Aware Multi-Human Behavior Prediction with Vision Language Models},
author = {Utsav Panchal and Yuchen Liu and Luigi Palmieri and Ilche Georgievski and Marco Aiello},
journal= {arXiv preprint arXiv:2512.15957},
year = {2025}
}
Comments
Accepted at IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2026