English

Teaching LLMs to Learn Tool Trialing and Execution through Environment Interaction

Software Engineering 2026-01-21 v1 Artificial Intelligence

Abstract

Equipping Large Language Models (LLMs) with external tools enables them to solve complex real-world problems. However, the robustness of existing methods remains a critical challenge when confronting novel or evolving tools. Existing trajectory-centric paradigms primarily rely on memorizing static solution paths during training, which limits the ability of LLMs to generalize tool usage to newly introduced or previously unseen tools. In this paper, we propose ToolMaster, a framework that shifts tool use from imitating golden tool-calling trajectories to actively learning tool usage through interaction with the environment. To optimize LLMs for tool planning and invocation, ToolMaster adopts a trial-and-execution paradigm, which trains LLMs to first imitate teacher-generated trajectories containing explicit tool trials and self-correction, followed by reinforcement learning to coordinate the trial and execution phases jointly. This process enables agents to autonomously explore correct tool usage by actively interacting with environments and forming experiential knowledge that benefits tool execution. Experimental results demonstrate that ToolMaster significantly outperforms existing baselines in terms of generalization and robustness across unseen or unfamiliar tools. All code and data are available at https://github.com/NEUIR/ToolMaster.

Keywords

Cite

@article{arxiv.2601.12762,
  title  = {Teaching LLMs to Learn Tool Trialing and Execution through Environment Interaction},
  author = {Xingjie Gao and Pengcheng Huang and Zhenghao Liu and Yukun Yan and Shuo Wang and Zulong Chen and Chen Qian and Ge Yu and Yu Gu},
  journal= {arXiv preprint arXiv:2601.12762},
  year   = {2026}
}
R2 v1 2026-07-01T09:10:06.652Z