SEAD: Competence-Aware On-Policy Distillation via Entropy-Guided Supervision
Abstract
On-policy distillation (OPD) has a property absent in offline distillation and RL: teacher supervision quality depends on student competence. Incoherent rollouts yield noisy gradients; already-mastered tokens yield redundant ones. This creates waste at three scales (tokens, training phases, and prompts) yet existing methods supervise uniformly. We introduce SEAD, which uses entropy as a unified probe of this competence-dependent degradation at three scales: (1) joint teacher-student entropy partitions tokens into zones receiving tailored divergences or zero gradient (approx. 50% skipped); (2) a cosine schedule anneals from forward to reverse KL as competence grows; (3) a competence-gated curriculum introduces prompts easy-to-hard. These components are symbiotically necessary: token selection requires coherent rollouts (curriculum), annealing requires monotonic improvement (also curriculum). On OLMo-3 (7B to 32B), SEAD achieves +4.8 avg accuracy over vanilla OPD across six math benchmarks, with ablations confirming super-additive interactions.
Keywords
Cite
@article{arxiv.2606.28562,
title = {SEAD: Competence-Aware On-Policy Distillation via Entropy-Guided Supervision},
author = {Chia-Hsuan Lee and Zelei Cheng and Yu Wang and Renkun Ni and Sambit Sahu and Shi-Xiong Zhang and William Campbell},
journal= {arXiv preprint arXiv:2606.28562},
year = {2026}
}