Classification Done Right for Vision-Language Pre-Training

Zilong Huang, Qinghao Ye, Bingyi Kang, Jiashi Feng, Haoqi Fan
2/3/2026

No official code found. Best implementation: x-cls/superclass (224 stars, Apache-2.0, last commit 2025-03-20).

Abstract

We introduce SuperClass, a super simple classification method for vision-language pre-training on image-text data. Unlike its contrastive counterpart CLIP who contrast with a text encoder, SuperClass directly utilizes tokenized raw text as supervised classification labels, without the need for additional text filtering or selection. Due to the absence of the text encoding as contrastive target, SuperClass does not require a text encoder and does not need to maintain a large batch size as CLIP does. SuperClass demonstrated superior performance on various downstream tasks, including classic computer vision benchmarks and vision language downstream tasks. We further explored the scaling behavior of SuperClass on model size, training length, or data size, and reported encouraging results and comparisons to CLIP. https://github.com/x-cls/superclass

View on arXivView PDF

Code Implementations(1)

x-cls/superclassCommunity100%
Apache-2.0

[NeurIPS 2024] Classification Done Right for Vision-Language Pre-Training

2246Shell, PythonSep 30, 20241 years agoApache-2.0
research

Similar Papers

CiT: Curation in Training for Effective Vision-Language Data

90% match1 repo772026

SuperCLIP: CLIP with Simple Classification Supervision

88% match2025

Enhancing Vision-Language Model with Unmasked Token Alignment

88% match1 repo112024

VeCAF: Vision-language Collaborative Active Finetuning with Training Objective Awareness

88% match1 repo522024

ViTamin: Designing Scalable Vision Models in the Vision-Language Era

88% match1 repo2122026

DRIP: Dynamic patch Reduction via Interpretable Pooling

88% match2025

Cite this paper

@article{huang2026classification,
  title  = {Classification Done Right for Vision-Language Pre-Training},
  author = {Zilong Huang and Qinghao Ye and Bingyi Kang and Jiashi Feng and Haoqi Fan},
  year   = {2026},
  eprint = {2411.03313},
  archivePrefix = {arXiv},
  doi    = {10.48550/arXiv.2411.03313},
  url    = {https://arxiv.org/abs/2411.03313},
  journal = {NEURIPS 2024 2024}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2411.03313.svg)](https://codeofpaper.com/paper/2411.03313)

Discussion