Autoregressive (AR) image generators offer a language-model-friendly approach to image generation by predicting discrete image tokens in a causal sequence. However, unlike diffusion models, AR models lack a mechanism to refine previous predictions, limiting their generation quality. In this paper, we introduce TensorAR, a new AR paradigm that reformulates image generation from next-token prediction to next-tensor prediction. By generating overlapping windows of image patches (tensors) in a sliding fashion, TensorAR enables iterative refinement of previously generated content. To prevent information leakage during training, we propose a discrete tensor noising scheme, which perturbs input tokens via codebook-indexed noise. TensorAR is implemented as a plug-and-play module compatible with existing AR models. Extensive experiments on LlamaGEN, Open-MAGVIT2, and RAR demonstrate that TensorAR significantly improves the generation performance of autoregressive models.
@article{arxiv.2505.16324,
title = {From Prediction to Perfection: Introducing Refinement to Autoregressive Image Generation},
author = {Cheng Cheng and Lin Song and Di An and Yicheng Xiao and Xuchong Zhang and Hongbin Sun and Ying Shan},
journal= {arXiv preprint arXiv:2505.16324},
year = {2026}
}