Class-Incremental Learning based on Label Generation

Yijia Shao, Yiduo Guo, Dongyan Zhao, Bing Liu
6/22/2023
cs.CLcs.AIcs.LG

Official code: shaoyijia/VAG (9 stars, last commit 2023-11-17).

Abstract

Despite the great success of pre-trained language models, it is still a challenge to use these models for continual learning, especially for the class-incremental learning (CIL) setting due to catastrophic forgetting (CF). This paper reports our finding that if we formulate CIL as a continual label generation problem, CF is drastically reduced and the generalizable representations of pre-trained models can be better retained. We thus propose a new CIL method (VAG) that also leverages the sparsity of vocabulary to focus the generation and creates pseudo-replay samples by using label semantics. Experimental results show that VAG outperforms baselines by a large margin.

View on arXivView PDF

Code Implementations(1)

shaoyijia/VAGOfficial68%

[ACL 2023] Class-Incremental Learning based on Label Generation

90Shell, PythonJun 7, 20232 years ago

Similar Papers

Dynamic Prompt Adjustment for Multi-Label Class-Incremental Learning

88% match2024

Balancing the Causal Effects in Class-Incremental Learning

86% match2024

Knowledge Restore and Transfer for Multi-Label Class-Incremental Learning

85% match1 repo122026

An Efficient Replay for Class-Incremental Learning with Pre-trained Models

85% match2024

Learning without Forgetting for Vision-Language Models

84% match1 repo642023

Hierarchical Semantic Tree Anchoring for CLIP-Based Class-Incremental Learning

84% match2025

Cite this paper

@article{shao2023classincremental,
  title  = {Class-Incremental Learning based on Label Generation},
  author = {Yijia Shao and Yiduo Guo and Dongyan Zhao and Bing Liu},
  year   = {2023},
  eprint = {2306.12619},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.12619v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.12619.svg)](https://codeofpaper.com/paper/2306.12619)

Discussion