English

ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation

Artificial Intelligence 2026-05-26 v2

Abstract

Skill-distillation pipelines learn reusable rules from LLM agent trajectories, but they lack a key signal: how much each step costs. Without per-step cost, a pipeline cannot distinguish adding a missing step to fix a bug from removing an expensive step that never affected the outcome. We use the cost-attribution gap to ask whether the rule types inside a distilled skill transfer the same way to new tasks. ClawTrace records cost-attributed agent traces and compiles each session into a TraceCard; CostCraft reads TraceCards and writes three kinds of skill patches: preserve, prune, and repair. We find a pattern aggregate metrics hide. On 30 held-out SpreadsheetBench tasks across two seeds, removing prune patches roughly tripled the quality-regression count without lowering median cost. Across the full 84-task SkillsBench transfer, CostCraft saves no aggregate cost. All three quality regressions trace to the preserve lane, and both quality wins trace to the prune lane: prune patches act as quality guardrails while preserve patches drive regressions. We argue that reusable agent skills should be evaluated at the rule-type level, not as monolithic instruction packages. To support this, we release ClawTrace, the TraceCard schema, and the full set of typed skills.

Cite

@article{arxiv.2604.23853,
  title  = {ClawTrace: Cost-Aware Tracing for LLM Agent Skill Distillation},
  author = {Boqin Yuan and Yue Su and Renchu Song and Sen Yang and Jing Qin},
  journal= {arXiv preprint arXiv:2604.23853},
  year   = {2026}
}

Comments

Accepted at Agent Skills '26 Workshop, ACM Conference on AI and Agentic Systems (CAIS 2026), San Jos\'e, CA, May 26, 2026

R2 v1 2026-07-01T12:36:00.355Z