As large language models (LLMs) are increasingly deployed in high-stakes settings, the risk of generating harmful or toxic content remains a central challenge. Post-hoc alignment methods are brittle: once unsafe patterns are learned during pretraining, they are hard to remove. In this work, we present a data-centric pretraining framework that builds safety into the model from the start. Our framework consists of four key steps: (i) Safety Filtering: building a safety classifier to classify webdata into safe and unsafe categories; (ii) Safety Rephrasing: we recontextualize unsafe webdata into safer narratives; (iii) Native Refusal: we develop RefuseWeb and Moral Education pretraining datasets that actively teach model to refuse on unsafe content and the moral reasoning behind it, and (iv) Harmfulness-Tag annotated pretraining: we flag unsafe content during pretraining using a special token, and use it to steer model away from unsafe generations at inference. Our safety-pretrained models reduce attack success rates from 38.8\% to 8.4\% on standard LLM safety benchmarks with no performance degradation on general tasks.
@article{arxiv.2504.16980,
title = {Safety Pretraining: Toward the Next Generation of Safe AI},
author = {Pratyush Maini and Sachin Goyal and Dylan Sam and Alex Robey and Yash Savani and Yiding Jiang and Andy Zou and Matt Fredrikson and Zacharcy C. Lipton and J. Zico Kolter},
journal= {arXiv preprint arXiv:2504.16980},
year = {2025}
}