Sparse autoencoders are a promising new approach for decomposing language model activations for interpretation and control. They have been applied successfully to vision transformer image encoders and to small-scale diffusion models. Inference-Time Decomposition of Activations (ITDA) is a recently proposed variant of dictionary learning that takes the dictionary to be a set of data points from the activation distribution and reconstructs them with gradient pursuit. We apply Sparse Autoencoders (SAEs) and ITDA to a large text-to-image diffusion model, Flux 1, and consider the interpretability of embeddings of both by introducing a visual automated interpretation pipeline. We find that SAEs accurately reconstruct residual stream embeddings and beat MLP neurons on interpretability. We are able to use SAE features to steer image generation through activation addition. We find that ITDA has comparable interpretability to SAEs.
@article{arxiv.2505.24360,
title = {Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning},
author = {Stepan Shabalin and Ayush Panda and Dmitrii Kharlapenko and Abdur Raheem Ali and Yixiong Hao and Arthur Conmy},
journal= {arXiv preprint arXiv:2505.24360},
year = {2025}
}
Comments
10 pages, 10 figures, Mechanistic Interpretability for Vision at CVPR 2025