HomeComputation & LanguagearXiv:2605.29927

Does The Way You Plan Matter? An Empirical Study of Planning Representations for LLM Web Agents

Abstract

Despite recent advances, LLM-based web agents still struggle with limited exploration, omission of critical steps, and sensitivity to task constraints. Prior work suggests that many of these failures stem from weaknesses in planning, yet the impact of alternative natural language plan representation remains unexplored. To address this, we introduce PlanAhead, a static planner-executor framework that evaluates the impact of plan representation in agent performance. We first automatically categorize WebArena tasks into 3 difficulty levels, enabling consistent difficulty grading without human annotation. Then we systematically evaluate 4 different plan representations on the tasks categorized as hard: sequential subgoals, narrative, pseudocode, and checklist; across different families of multimodal LLM powered agents (OpenAI, Alibaba, and Google). To account for stochastic variability, we introduce two novel evaluation metrics: Achievement Rate (AR) and Solved-Task Consistency (STC). Our results show that both, the plan formulation and the underlying LLM generating the plan, significantly influence web-agent robustness and task success.

Comments: Extended version of paper submitted to EMNLP, waiting for acceptance

Cite

@article{arxiv.2605.29927,
  title  = {Does The Way You Plan Matter? An Empirical Study of Planning Representations for LLM Web Agents},
  author = {Alejandra Zambrano and Sara Vera Marjanovic and Imene Kerboua and Xing Han Lù and Leila Kosseim},
  journal= {arXiv preprint arXiv:2605.29927},
  year   = {2026}
}