English

Attention in Diffusion Model: A Survey

Machine Learning 2025-04-08 v1 Artificial Intelligence Computer Vision and Pattern Recognition

Abstract

Attention mechanisms have become a foundational component in diffusion models, significantly influencing their capacity across a wide range of generative and discriminative tasks. This paper presents a comprehensive survey of attention within diffusion models, systematically analysing its roles, design patterns, and operations across different modalities and tasks. We propose a unified taxonomy that categorises attention-related modifications into parts according to the structural components they affect, offering a clear lens through which to understand their functional diversity. In addition to reviewing architectural innovations, we examine how attention mechanisms contribute to performance improvements in diverse applications. We also identify current limitations and underexplored areas, and outline potential directions for future research. Our study provides valuable insights into the evolving landscape of diffusion models, with a particular focus on the integrative and ubiquitous role of attention.

Keywords

Cite

@article{arxiv.2504.03738,
  title  = {Attention in Diffusion Model: A Survey},
  author = {Litao Hua and Fan Liu and Jie Su and Xingyu Miao and Zizhou Ouyang and Zeyu Wang and Runze Hu and Zhenyu Wen and Bing Zhai and Yang Long and Haoran Duan and Yuan Zhou},
  journal= {arXiv preprint arXiv:2504.03738},
  year   = {2025}
}
R2 v1 2026-06-28T22:47:25.405Z