LLM role-playing, i.e., using LLMs to simulate specific personas, has emerged as a key capability in various applications, such as companionship, content creation and digital games. While current models effectively capture character tones and knowledge, simulating the inner thoughts behind their behaviors remains a challenge. Towards cognitive simulation in LLM role-play, previous efforts mainly suffer from two deficiencies: lacking data with high-quality reasoning traces, and lacking reliable reward signals aligned with human preferences. In this paper, we propose HER, a unified framework for cognitive-level persona simulation. HER introduces dual-layer thinking, which distinguishes characters' first-person thinking from LLMs' third-person thinking. To bridge these gaps, we curate reasoning-augmented role-playing data via reverse engineering, and construct human-aligned principles and reward models. Leveraging these resources, we train HER models based on Qwen3-32B via supervised and reinforcement learning. Extensive experiments validate the effectiveness of our approach. Notably, our models significantly outperform the Qwen3-32B baseline, achieving a 30.26 improvement on the CoSER benchmark and a 14.97% gain on the Minimax Role-Play Bench. Our datasets, principles, and models are released to facilitate future research.
@article{arxiv.2601.21459,
title = {HER: Human-like Reasoning and Reinforcement Learning for LLM Role-playing},
author = {Chengyu Du and Xintao Wang and Aili Chen and Weiyuan Li and Rui Xu and Junteng Liu and Zishan Huang and Rong Tian and Zijun Sun and Yuhao Li and Liheng Feng and Deming Ding and Pengyu Zhao and Yanghua Xiao},
journal= {arXiv preprint arXiv:2601.21459},
year = {2026}
}