We introduce SENT-Map, a semantically enhanced topological map for representing indoor environments, designed to support autonomous navigation and manipulation by leveraging advancements in foundational models (FMs). Through representing the environment in a JSON text format, we enable semantic information to be added and edited in a format that both humans and FMs understand, while grounding the robot to existing nodes during planning to avoid infeasible states during deployment. Our proposed framework employs a two stage approach, first mapping the environment alongside an operator with a Vision-FM, then using the SENT-Map representation alongside a natural-language query within an FM for planning. Our experimental results show that semantic-enhancement enables even small locally-deployable FMs to successfully plan over indoor environments.
@article{arxiv.2511.03165,
title = {SENT Map -- Semantically Enhanced Topological Maps with Foundation Models},
author = {Raj Surya Rajendran Kathirvel and Zach A Chavis and Stephen J. Guy and Karthik Desingh},
journal= {arXiv preprint arXiv:2511.03165},
year = {2025}
}
Comments
Accepted at ICRA 2025 Workshop on Foundation Models and Neuro-Symbolic AI for Robotics