一张图像相当于 16x16 个词：用于大规模图像识别的 Transformer
计算机视觉与模式识别
2021-06-04 v2 人工智能
机器学习
摘要
虽然 Transformer 架构已成为自然语言处理任务的事实标准，但其在计算机视觉中的应用仍然有限。在视觉领域，注意力要么与卷积网络结合使用，要么用于替换卷积网络的某些组件同时保持其整体结构不变。我们表明这种对 CNN 的依赖是不必要的，直接应用于图像块序列的纯 transformer 可以在图像分类任务上表现非常好。当在大量数据上预训练并迁移到多个中型或小型图像识别基准（ImageNet、CIFAR-100、VTAB 等）时，Vision Transformer (ViT) 相较于最先进的卷积网络取得了优异的结果，同时训练所需的计算资源大幅减少。
引用
@article{arxiv.2010.11929,
title = {An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale},
author = {Alexey Dosovitskiy and Lucas Beyer and Alexander Kolesnikov and Dirk Weissenborn and Xiaohua Zhai and Thomas Unterthiner and Mostafa Dehghani and Matthias Minderer and Georg Heigold and Sylvain Gelly and Jakob Uszkoreit and Neil Houlsby},
journal= {arXiv preprint arXiv:2010.11929},
year = {2021}
}
备注
Fine-tuning code and pre-trained models are available at https://github.com/google-research/vision_transformer. ICLR camera-ready version with 2 small modifications: 1) Added a discussion of CLS vs GAP classifier in the appendix, 2) Fixed an error in exaFLOPs computation in Figure 5 and Table 6 (relative performance of models is basically not affected)