English

Inference-Time Diversity in RL-Trained Lean Theorem Provers: A Diagnostic Study

Artificial Intelligence 2026-05-19 v2

Abstract

RL-trained Lean theorem provers mode-collapse at inference time: on miniF2F-test with DeepSeek-Prover-V1.5-RL, doubling the i.i.d.\ sampling budget from k=32k{=}32 to k=64k{=}64 produces zero additional solved theorems (42/244 in both cases). A fixed schedule of 15 tactic skeletons breaks this plateau and recovers a +45+45% relative improvement at k=16k{=}16 (mean Δ=+12.3±4.2\Delta = +12.3 \pm 4.2 theorems across n=3n{=}3 seeds, sign preserved in every seed). A controlled diversity ablation rules out the prompt-diversity confound: tactic skeletons help, paraphrases match the baseline, and irrelevant Lean comments actively degrade. A leave-one-out formalization-difficulty stratification reveals a structural-content gradient across the three perturbations. The phenomenon is RL-specific: V1.5-Base proves zero theorems regardless of intervention, identifying RL as the stage that creates the proof capability which subsequently collapses; extending to two additional 7B Lean provers, RL-trained DeepSeek-Prover-V2-7B contributes +3+3 frontier solves no i.i.d.\ baseline can reach despite a flat aggregate, while SFT-trained Goedel-Prover does not (10.0±4.4-10.0 \pm 4.4 theorems, n=3n{=}3, sign preserved every seed). Inference-time structural diversity is a cheap, complementary axis for RL-trained provers, orthogonal to scaling model size or training compute.

Keywords

Cite

@article{arxiv.2601.16172,
  title  = {Inference-Time Diversity in RL-Trained Lean Theorem Provers: A Diagnostic Study},
  author = {Zachary Burton},
  journal= {arXiv preprint arXiv:2601.16172},
  year   = {2026}
}

Comments

20 pages

R2 v1 2026-07-01T09:16:12.452Z