English

Peak-Then-Collapse and the Four Interface Channels of Knowledge-Graph Tool Use

Computation and Language 2026-05-26 v1

Abstract

We test the standard RLVR tool-use recipe -- GRPO on Qwen2.5-7B-Instruct -- on a deliberately minimal knowledge-graph tool API: four Freebase navigation verbs over Complex WebQuestions. Under a self-verifiable retrieval reward, the policy's tool-grounded answer rate climbs from 3.8%3.8\% to 9.6%9.6\% over 250 steps, then collapses to 0%0\% within a single 50-step window -- a \emph{peak-then-collapse} pattern replicated across four seeds. Across seven reward designs, we find four recurring failure modes: adding denser or more targeted proxy rewards shifts the failure mode rather than eliminating it. We argue that a key difference from Python interpreters, web search, and JSON APIs is interface feedback: their failures often leak natural-language signal the model saw in pretraining. A Python traceback names the failing line; an empty Freebase result \texttt{[]} does not. Stripping away that surface exposes a degradation regime that same-family reward redesigns do not fix. A direct oracle ablation rules out relation selection: injecting gold relations at every retrieval call lifts exact-match accuracy by only +0.20+0.20~pp, and 95.4%95.4\% of retrieval-dependent errors are retrieval-composition failures rather than answer-extraction failures. As a mitigation, one-iteration self-distillation reaches 40.0%40.0\% EM at 7B and is capacity-invariant: doubling capacity to 14B improves EM by only 0.250.25~pp, and initialization barely matters -- the ceiling appears interface-bound within the 7B--14B range tested.

Cite

@article{arxiv.2605.26037,
  title  = {Peak-Then-Collapse and the Four Interface Channels of Knowledge-Graph Tool Use},
  author = {Tianda Sun and Dimitar Kazakov},
  journal= {arXiv preprint arXiv:2605.26037},
  year   = {2026}
}

Comments

18 pages, 9 figures

R2 v1 2026-07-22T07:32:53.165Z