English

Delta-SVD: Efficient Compression for Personalized Text-to-Image Models

Computer Vision and Pattern Recognition 2025-08-26 v1

Abstract

Personalized text-to-image models such as DreamBooth require fine-tuning large-scale diffusion backbones, resulting in significant storage overhead when maintaining many subject-specific models. We present Delta-SVD, a post-hoc, training-free compression method that targets the parameter weights update induced by DreamBooth fine-tuning. Our key observation is that these delta weights exhibit strong low-rank structure due to the sparse and localized nature of personalization. Delta-SVD first applies Singular Value Decomposition (SVD) to factorize the weight deltas, followed by an energy-based rank truncation strategy to balance compression efficiency and reconstruction fidelity. The resulting compressed models are fully plug-and-play and can be re-constructed on-the-fly during inference. Notably, the proposed approach is simple, efficient, and preserves the original model architecture. Experiments on a multiple subject dataset demonstrate that Delta-SVD achieves substantial compression with negligible loss in generation quality measured by CLIP score, SSIM and FID. Our method enables scalable and efficient deployment of personalized diffusion models, making it a practical solution for real-world applications that require storing and deploying large-scale subject customizations.

Keywords

Cite

@article{arxiv.2508.16863,
  title  = {Delta-SVD: Efficient Compression for Personalized Text-to-Image Models},
  author = {Tangyuan Zhang and Shangyu Chen and Qixiang Chen and Jianfei Cai},
  journal= {arXiv preprint arXiv:2508.16863},
  year   = {2025}
}
R2 v1 2026-07-01T05:02:36.106Z