OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models

Anas Awadalla; Irena Gao; Josh Gardner; Jack Hessel; Yusuf Hanafy; Wanrong Zhu; Kalyani Marathe; Yonatan Bitton; Samir Gadre; Shiori Sagawa; Jenia Jitsev; Simon Kornblith; Pang Wei Koh; Gabriel Ilharco; Mitchell Wortsman; Ludwig Schmidt

OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models

Computer Vision and Pattern Recognition 2023-08-08 v2 Artificial Intelligence Machine Learning

Authors: Anas Awadalla , Irena Gao , Josh Gardner , Jack Hessel , Yusuf Hanafy , Wanrong Zhu , Kalyani Marathe , Yonatan Bitton , Samir Gadre , Shiori Sagawa , Jenia Jitsev , Simon Kornblith , Pang Wei Koh , Gabriel Ilharco , Mitchell Wortsman , Ludwig Schmidt

View on arXiv ↗ PDF ↗

Abstract

We introduce OpenFlamingo, a family of autoregressive vision-language models ranging from 3B to 9B parameters. OpenFlamingo is an ongoing effort to produce an open-source replication of DeepMind's Flamingo models. On seven vision-language datasets, OpenFlamingo models average between 80 - 89% of corresponding Flamingo performance. This technical report describes our models, training data, hyperparameters, and evaluation suite. We share our models and code at https://github.com/mlfoundations/open_flamingo.

Keywords

large language model code generation

Cite

@article{arxiv.2308.01390,
  title  = {OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models},
  author = {Anas Awadalla and Irena Gao and Josh Gardner and Jack Hessel and Yusuf Hanafy and Wanrong Zhu and Kalyani Marathe and Yonatan Bitton and Samir Gadre and Shiori Sagawa and Jenia Jitsev and Simon Kornblith and Pang Wei Koh and Gabriel Ilharco and Mitchell Wortsman and Ludwig Schmidt},
  journal= {arXiv preprint arXiv:2308.01390},
  year   = {2023}
}

OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models

Abstract

Keywords

Cite

Related papers