English

When Independent Sampling Outperforms Agentic Reasoning

Machine Learning 2026-05-12 v1

Abstract

We study how to allocate inference-time compute for competitive programming under fixed budgets. Evaluating 216 Codeforces problems across Divisions 1-3, we compare agent-based reasoning with repeated independent sampling (k-shot) as a function of both cost and number of model calls. Across models and difficulty levels, k-shot consistently achieves a better accuracy-cost and accuracy-query tradeoff. This gap persists despite prompt caching in agent frameworks, indicating lower per-call effectiveness. Our results show that, for self-contained algorithmic tasks, independent exploration can outperform deeper agentic reasoning under realistic resource constraints. We also provide a budget-allocation analysis when the inference budget is fixed, and prove that a cost-optimal solver minimizes the principled metric log failure likelihood per dollar.

Keywords

Cite

@article{arxiv.2605.08478,
  title  = {When Independent Sampling Outperforms Agentic Reasoning},
  author = {Yihe Dong and Boris Shigida},
  journal= {arXiv preprint arXiv:2605.08478},
  year   = {2026}
}
R2 v1 2026-07-01T12:59:05.617Z