SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?

Hasan Abed Al Kader Hammoud; Hani Itani; Fabio Pizzati; Philip Torr; Adel Bibi; Bernard Ghanem

SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?

Computer Vision and Pattern Recognition 2024-07-19 v2 Artificial Intelligence Machine Learning

Authors: Hasan Abed Al Kader Hammoud , Hani Itani , Fabio Pizzati , Philip Torr , Adel Bibi , Bernard Ghanem

Abstract

We present SynthCLIP, a CLIP model trained on entirely synthetic text-image pairs. Leveraging recent text-to-image (TTI) networks and large language models (LLM), we generate synthetic datasets of images and corresponding captions at scale, with no human intervention. In this work, we provide an analysis on CLIP models trained on synthetic data. We provide insights on the data generation strategy, number of samples required, scaling trends, and resulting properties. We also introduce SynthCI-30M, a purely synthetic dataset comprising 30 million captioned images. Our code, trained models, and data, are released as open source at https://github.com/hammoudhasan/SynthCLIP

Keywords

contrastive language-image pre-training synthetic data generation image processing

Cite

@article{arxiv.2402.01832,
  title  = {SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?},
  author = {Hasan Abed Al Kader Hammoud and Hani Itani and Fabio Pizzati and Philip Torr and Adel Bibi and Bernard Ghanem},
  journal= {arXiv preprint arXiv:2402.01832},
  year   = {2024}
}

Comments

Under review

SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?

Abstract

Keywords

Cite

Comments

Related papers