English

A Unified Framework for Diffusion Model Unlearning with f-Divergence

Machine Learning 2026-05-27 v2 Computer Vision and Pattern Recognition

Abstract

Most existing methods for concept unlearning in text-to-image diffusion models minimize a mean squared error (MSE) loss between the denoiser outputs conditioned on a target and an anchor concept, which is implicitly the KL divergence between two Gaussians. We generalize this objective to any ff-divergence, recovering MSE as the KL instance, and identify a family of α\alpha-divergences whose Gaussian closed-form yields cheap, MSE-like training objectives. For the remaining ff-divergences, we provide a min-max objective based on the variational formulation of the ff-divergence. We theoretically analyze and numerically validate how different ff-divergences impact the gradient magnitude and the convergence properties of the algorithm, affecting the quality of unlearning. For instance, we observe that the Hellinger closed-form instance consistently dominates MSE across multiple scenarios. More generally, the proposed unified framework offers a flexible paradigm for selecting the optimal divergence based on the application and user goal, allowing for finer control over the trade-off between unlearning efficacy and generative fidelity.

Keywords

Cite

@article{arxiv.2509.21167,
  title  = {A Unified Framework for Diffusion Model Unlearning with f-Divergence},
  author = {Nicola Novello and Federico Fontana and Luigi Cinque and Deniz Gunduz and Andrea M. Tonello},
  journal= {arXiv preprint arXiv:2509.21167},
  year   = {2026}
}

Comments

Accepted at ICML 2026

R2 v1 2026-07-01T05:56:12.228Z