SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?

Hasan Abed Al Kader Hammoud, Hani Itani, Fabio Pizzati, Philip Torr, Adel Bibi +1 more
2/2/2024
cs.CVcs.AIcs.LG

Official code: hammoudhasan/SynthCLIP (104 stars, last commit 2025-03-23).

Abstract

We present SynthCLIP, a CLIP model trained on entirely synthetic text-image pairs. Leveraging recent text-to-image (TTI) networks and large language models (LLM), we generate synthetic datasets of images and corresponding captions at scale, with no human intervention. In this work, we provide an analysis on CLIP models trained on synthetic data. We provide insights on the data generation strategy, number of samples required, scaling trends, and resulting properties. We also introduce SynthCI-30M, a purely synthetic dataset comprising 30 million captioned images. Our code, trained models, and data, are released as open source at https://github.com/hammoudhasan/SynthCLIP

View on arXivView PDF

Code Implementations(1)

hammoudhasan/SynthCLIPOfficial100%

Code base of SynthCLIP: CLIP training with purely synthetic text-image pairs from LLMs and TTIs.

1042Shell, PythonJan 21, 20241 years ago

Similar Papers

Synthetic Image Detection with CLIP: Understanding and Assessing Predictive Cues

84% match2026

LLM2CLIP: Powerful Language Model Unlocks Richer Visual Representation

83% match1 repo6782024

Synth$^2$: Boosting Visual-Language Models with Synthetic Captions and Image Embeddings

83% match2024

LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts

83% match1 repo112024

VeCLIP: Improving CLIP Training via Visual-enriched Captions

82% match1 repo2562023

CapsFusion: Rethinking Image-Text Data at Scale

82% match1 repo2152026

Cite this paper

@article{hammoud2024synthclip,
  title  = {SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?},
  author = {Hasan Abed Al Kader Hammoud and Hani Itani and Fabio Pizzati and Philip Torr and Adel Bibi and Bernard Ghanem},
  year   = {2024},
  eprint = {2402.01832},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2402.01832v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.01832.svg)](https://codeofpaper.com/paper/2402.01832)

Discussion