English

Prompt-aware classifier free guidance for diffusion models

Sound 2025-10-07 v2 Artificial Intelligence Multimedia Audio and Speech Processing

Abstract

Diffusion models have achieved remarkable progress in image and audio generation, largely due to Classifier-Free Guidance. However, the choice of guidance scale remains underexplored: a fixed scale often fails to generalize across prompts of varying complexity, leading to oversaturation or weak alignment. We address this gap by introducing a prompt-aware framework that predicts scale-dependent quality and selects the optimal guidance at inference. Specifically, we construct a large synthetic dataset by generating samples under multiple scales and scoring them with reliable evaluation metrics. A lightweight predictor, conditioned on semantic embeddings and linguistic complexity, estimates multi-metric quality curves and determines the best scale via a utility function with regularization. Experiments on MSCOCO~2014 and AudioCaps show consistent improvements over vanilla CFG, enhancing fidelity, alignment, and perceptual preference. This work demonstrates that prompt-aware scale selection provides an effective, training-free enhancement for pretrained diffusion backbones.

Keywords

Cite

@article{arxiv.2509.22728,
  title  = {Prompt-aware classifier free guidance for diffusion models},
  author = {Xuanhao Zhang and Chang Li},
  journal= {arXiv preprint arXiv:2509.22728},
  year   = {2025}
}

Comments

6 pages, 3 figures

R2 v1 2026-07-01T05:59:32.186Z