Role-playing with large language models is fundamentally a session-level task, requiring agents to sustain character identity and interaction quality across extended multi-turn conversations. Yet existing evaluation and optimization methods remain largely turn-level, failing to capture long-horizon quality. We propose DynSess, a unified session-level framework for role-playing agents. DynSess-Eval scores complete dialogue sessions via rubrics targeting long-horizon behaviors. Leveraging its session-level rewards, we construct high-quality training trajectories through multi-turn lookahead search and train DynSess-Character with two complementary variants: DSPO (off-policy) and GSRPO (on-policy). Experiments show that DynSess-Eval aligns with human judgments substantially better than prior evaluators, and blind human evaluation further shows that DynSess-Character matches the strongest character model despite using substantially fewer parameters, while maintaining strong role consistency and interactive ability. Our dataset and code will be released to facilitate future research.
@article{arxiv.2605.29256,
title = {DynSess: Dynamic Session-Level Evaluation and Optimization Framework for Role-Playing Agents},
author = {Rongsheng Zhang and Jiji Tang and Junnan Ren and Zuyi Bao and Weijie Chen and Ruofan Hu and Zhou Zhao and Tangjie Lv and Yan Zhang},
journal= {arXiv preprint arXiv:2605.29256},
year = {2026}
}