Optimus: A Robust Defense Framework for Mitigating Toxicity while Fine-Tuning Conversational AI
Abstract
Customizing Large Language Models (LLMs) on untrusted datasets poses severe risks of injecting toxic behaviors. In this work, we introduce Optimus, a novel defense framework designed to mitigate fine-tuning harms while preserving conversational utility. Unlike existing defenses that rely heavily on precise toxicity detection or restrictive filtering, Optimus addresses the critical challenge of ensuring robust mitigation even when toxicity classifiers are imperfect or biased. Optimus integrates a training-free toxicity classification scheme that repurposes the safety alignment of commodity LLMs, and employs a dual-strategy alignment process combining synthetic "healing data" with Direct Preference Optimization (DPO) to efficiently steer models toward safety. Extensive evaluations demonstrate that Optimus mitigates toxicity even when relying on extremely biased classifiers (with up to 85% degradation in Recall). Optimus outperforms the state-of-the-art defense StarDSS and exhibits strong resilience against adaptive adversarial and jailbreak attacks. Our source code and datasets are available at https://github.com/secml-lab-vt/Optimus
Cite
@article{arxiv.2507.05660,
title = {Optimus: A Robust Defense Framework for Mitigating Toxicity while Fine-Tuning Conversational AI},
author = {Aravind Cheruvu and Shravya Kanchi and Sifat Muhammad Abdullah and Nicholas Ka-Shing Kong and Daphne Yao and Murtuza Jadliwala and Bimal Viswanath},
journal= {arXiv preprint arXiv:2507.05660},
year = {2026}
}
Comments
Accepted at ACM CODASPY 2026