Efficient Post-training of LLMs for Code Generation With Offline Reinforcement Learning

Mingze Wu; Abhinav Anand; Shweta Verma; Mira Mezini

Efficient Post-training of LLMs for Code Generation With Offline Reinforcement Learning

Artificial Intelligence 2026-05-28 v1

Authors: Mingze Wu , Abhinav Anand , Shweta Verma , Mira Mezini

Abstract

Post-training using online reinforcement learning (RL) is an important training step for LLMs, including code-generating models. However, online RL for code generation involves LLM inference and verification of the generated output, which can take considerable time and resources. In this paper, we explore the application of offline RL to code-generating models by leveraging existing code datasets. Our experiments demonstrate that offline RL is an effective training strategy for improving LLM performance. We show that offline RL can be especially beneficial for small LLMs and challenging coding problems.

Keywords

code generation reinforcement learning

Cite

@article{arxiv.2605.28409,
  title  = {Efficient Post-training of LLMs for Code Generation With Offline Reinforcement Learning},
  author = {Mingze Wu and Abhinav Anand and Shweta Verma and Mira Mezini},
  journal= {arXiv preprint arXiv:2605.28409},
  year   = {2026}
}

Related papers

View all related →