HomeArtificial IntelligencearXiv:2605.29697

Beyond Trajectory Rewards: Step-level Credit Assignment for Agentic Search via Graph Modeling

Artificial Intelligence2026-05v1license

Abstract

In Agentic Search, trajectory-level outcome rewards fail to quantify the behavioral contributions of individual steps, while existing step-level reward methods typically rely on costly tree sampling. We view world knowledge as a latent world graph and each IS task as search within a latent task graph, where effective steps should make graph progress toward the answer node. Based on this prior, we propose Graph-Distance Contribution Reward (GDCR), a step-level process reward that scores newly-retrieved and newly-cited entities by their distance to the answer node in a training-time Entity-Relation (ER) graph. We further propose Step Advantage Policy Optimization (SAPO), which converts GDCR into step-level advantages and combines them with trajectory-level outcome advantages. Experiments on four challenging benchmarks validate the effectiveness of our method.

Comments: 15 pages, 8 figures

Cite

@article{arxiv.2605.29697,
  title  = {Beyond Trajectory Rewards: Step-level Credit Assignment for Agentic Search via Graph Modeling},
  author = {Yuchen Liu and Yingjie Feng and Lixiong Qin and Jiasi Chen and Jianing Yu and Sheng Gao and Sheng Yang and Weiran Xu},
  journal= {arXiv preprint arXiv:2605.29697},
  year   = {2026}
}