CogView: Mastering Text-to-Image Generation via Transformers

Ming Ding; Zhuoyi Yang; Wenyi Hong; Wendi Zheng; Chang Zhou; Da Yin; Junyang Lin; Xu Zou; Zhou Shao; Hongxia Yang; Jie Tang

CogView: Mastering Text-to-Image Generation via Transformers

Computer Vision and Pattern Recognition 2021-11-08 v3 Machine Learning

Authors: Ming Ding , Zhuoyi Yang , Wenyi Hong , Wendi Zheng , Chang Zhou , Da Yin , Junyang Lin , Xu Zou , Zhou Shao , Hongxia Yang , Jie Tang

View on arXiv ↗ PDF ↗

Abstract

Text-to-Image generation in the general domain has long been an open problem, which requires both a powerful generative model and cross-modal understanding. We propose CogView, a 4-billion-parameter Transformer with VQ-VAE tokenizer to advance this problem. We also demonstrate the finetuning strategies for various downstream tasks, e.g. style learning, super-resolution, text-image ranking and fashion design, and methods to stabilize pretraining, e.g. eliminating NaN losses. CogView achieves the state-of-the-art FID on the blurred MS COCO dataset, outperforming previous GAN-based models and a recent similar work DALL-E.

Keywords

image generation text-to-image generation vision transformer

Cite

@article{arxiv.2105.13290,
  title  = {CogView: Mastering Text-to-Image Generation via Transformers},
  author = {Ming Ding and Zhuoyi Yang and Wenyi Hong and Wendi Zheng and Chang Zhou and Da Yin and Junyang Lin and Xu Zou and Zhou Shao and Hongxia Yang and Jie Tang},
  journal= {arXiv preprint arXiv:2105.13290},
  year   = {2021}
}

Comments

to appear in NeurIPS 2021

CogView: Mastering Text-to-Image Generation via Transformers

Abstract

Keywords

Cite

Comments

Related papers