English

Forget Many, Forget Right: Scalable and Precise Concept Unlearning in Diffusion Models

Machine Learning 2026-05-19 v4 Computer Vision and Pattern Recognition

Abstract

Text-to-image diffusion models have achieved remarkable progress, yet their use raises copyright and misuse concerns, prompting research into machine unlearning. However, extending multi-concept unlearning to large-scale scenarios remains difficult due to three challenges: (i) conflicting weight updates that hinder unlearning or degrade generation; (ii) imprecise mechanisms that cause collateral damage to similar content; and (iii) reliance on additional data or modules, creating scalability bottlenecks. To address these, we propose Scalable-Precise Concept Unlearning (ScaPre), a unified framework tailored for large-scale unlearning. ScaPre introduces a conflict-aware stable design, integrating spectral trace regularization and geometry alignment to stabilize optimization, suppress conflicts, and preserve global structure. Furthermore, an Informax Decoupler identifies concept-relevant parameters and adaptively reweights updates, strictly confining unlearning to the target subspace. ScaPre yields an efficient closed-form solution without requiring auxiliary data or sub-models. Comprehensive experiments on objects, styles, and explicit content demonstrate that ScaPre effectively removes target concepts while maintaining generation quality. It forgets up to ×5\times \mathbf{5} more concepts than the best baseline within acceptable quality limits, achieving state-of-the-art precision and efficiency for large-scale unlearning. Code is available at https://github.com/kaiyuan02415/scapre

Keywords

Cite

@article{arxiv.2601.06162,
  title  = {Forget Many, Forget Right: Scalable and Precise Concept Unlearning in Diffusion Models},
  author = {Kaiyuan Deng and Gen Li and Yang Xiao and Bo Hui and Xiaolong Ma},
  journal= {arXiv preprint arXiv:2601.06162},
  year   = {2026}
}

Comments

Accepted at ICLR 2026

R2 v1 2026-07-01T08:58:17.923Z