Large Language Models (LLMs) and Vision Language Models (VLMs) have become popular tools for embodied high-level planning. However, their deployment in black-box settings often leads to unpredictable or costly errors. To harness their capabilities more reliably in robotic systems, we empirically investigate practical strategies for integrating language models as closed-loop planners. Concretely, we study how the control horizon and warm-starting impact the performance of language model-based planners. We design and conduct controlled experiments to extract actionable insights, providing recommendations that can help improve the performance and robustness of language model-based embodied planning. The full implementation and experiments are available on the project website
@article{arxiv.2511.07410,
title = {Using Language Models as Closed-Loop High-Level Planners for Robotics Applications: A Brief Overview and Benchmarks},
author = {Hao Wang and Sathwik Karnik and Bea Lim and Somil Bansal},
journal= {arXiv preprint arXiv:2511.07410},
year = {2026}
}