English

Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain

Machine Learning 2026-05-19 v2 Artificial Intelligence Computation and Language Information Theory math.IT

Abstract

Large language models (LLMs) make it plausible to build systems that improve through self-evolving loops, but many existing proposals are better understood as self-play and often plateau quickly. A central failure mode is that the loop synthesises more data without increasing learnable information for the next iteration. Through experiments on a self-play coding task, we reveal that sustainable self-evolution requires a self-synthesised data pipeline with learnable information that increases across iterations. We identify triadic roles that self-evolving LLMs play: the Proposer, which generates tasks; the Solver, which attempts solutions; and the Verifier, which provides training signals, and we identify three system designs that jointly target learnable information gain from this triadic roles perspective. Asymmetric co-evolution closes a weak-to-strong-to-weak loop across roles. Capacity growth expands parameter and inference-time budgets to match rising learnable information. Proactive information seeking introduces external context and new task sources that prevent saturation. Together, these modules provide a measurable, system-level path from brittle self-play dynamics to sustained self-evolution.

Keywords

Cite

@article{arxiv.2603.02218,
  title  = {Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain},
  author = {Wei Liu and Siya Qi and Yali Du and Yulan He},
  journal= {arXiv preprint arXiv:2603.02218},
  year   = {2026}
}

Comments

10 pages, 6 figures, 7 formulas, accepted by ICML 2026 position paper track

R2 v1 2026-07-01T10:59:47.090Z