HomeMachine LearningarXiv:2605.29582

PEARL: Training Socratic Tutors with Pedagogically Aligned Reinforcement Learning

Abstract

Large Language Models (LLMs) have shown promise as educational tutors, yet effective tutoring requires more than solving problems: it must provide progressive Socratic guidance and balance multiple pedagogical objectives across multi-turn interactions. However, training such tutors remains challenging due to limited-fidelity and weakly controllable student simulation, under-specified pedagogical reward modeling, and unstable multi-objective optimization. To overcome these limitations, we propose PEARL, a pedagogically aligned reinforcement learning framework for training Socratic tutoring agents, consisting of three key components. First, we introduce a controllable student simulator that decouples latent cognitive states from response generation to model diverse abilities and misconceptions. Second, we develop a generative reward model that jointly evaluates pedagogical quality and objective correctness for policy optimization. Finally, we propose a stable multi-objective RL scheme that discretizes rewards within each dimension and aggregates normalized advantages across dimensions, preventing high-variance objectives from dominating updates. Experiments on multiple benchmarks show that PEARL achieves the best performance among open-source models and remains competitive with leading proprietary LLMs, despite using only a 30B policy model.

Comments: 16 pages, 7 figures

Cite

@article{arxiv.2605.29582,
  title  = {PEARL: Training Socratic Tutors with Pedagogically Aligned Reinforcement Learning},
  author = {Qikai Chang and Zhenrong Zhang and Linbo Chen and Pengfei Hu and Jianshu Zhang and Youhui Guo and Jun Du},
  journal= {arXiv preprint arXiv:2605.29582},
  year   = {2026}
}