Fine-tuning enables large language models (LLMs) to adapt to specific domains, but often compromises their previously established safety alignment. To mitigate the degradation of model safety during fine-tuning, we introduce LookAhead Tuning, a lightweight and effective data-driven approach that preserves safety during fine-tuning. The method introduces two simple strategies that modify training data by previewing partial answer prefixes, thereby minimizing perturbations to the model's initial token distributions and maintaining its built-in safety mechanisms. Comprehensive experiments demonstrate that LookAhead Tuning effectively maintains model safety without sacrificing robust performance on downstream tasks. Our findings position LookAhead Tuning as a reliable and efficient solution for the safe and effective adaptation of LLMs.
@article{arxiv.2503.19041,
title = {LookAhead Tuning: Safer Language Models via Partial Answer Previews},
author = {Kangwei Liu and Mengru Wang and Yujie Luo and Lin Yuan and Mengshu Sun and Lei Liang and Zhiqiang Zhang and Jun Zhou and Bryan Hooi and Shumin Deng},
journal= {arXiv preprint arXiv:2503.19041},
year = {2025}
}