Autonomous robots operating in dynamic environments should identify and report anomalies. Embodying proactive mitigation improves safety and operational continuity. This paper presents a multimodal anomaly detection and mitigation system that integrates vision-language models and large language models to identify and report hazardous situations and conflicts in real-time. The proposed system enables robots to perceive, interpret, report, and if possible respond to urban and environmental anomalies through proactive detection mechanisms and automated mitigation actions. A key contribution in this paper is the integration of Hazardous and Conflict states into the robot's decision-making framework, where each anomaly type can trigger specific mitigation strategies. User studies (n = 30) demonstrated the effectiveness of the system in anomaly detection with 91.2% prediction accuracy and relatively low latency response times using edge-ai architecture.
@article{arxiv.2509.06768,
title = {Embodied Hazard Mitigation using Vision-Language Models for Autonomous Mobile Robots},
author = {Oluwadamilola Sotomi and Devika Kodi and Kiruthiga Chandra Shekar and Aliasghar Arab},
journal= {arXiv preprint arXiv:2509.06768},
year = {2025}
}