Applying ViT in Generalized Few-shot Semantic Segmentation

Liyuan Geng, Jinhong Xia, Yuanhe Guo
8/27/2024
cs.CV

No official code found. Best implementation: lygeng0427/ViTSeg (6 stars, MIT, last commit 2023-12-19).

Abstract

This paper explores the capability of ViT-based models under the generalized few-shot semantic segmentation (GFSS) framework. We conduct experiments with various combinations of backbone models, including ResNets and pretrained Vision Transformer (ViT)-based models, along with decoders featuring a linear classifier, UPerNet, and Mask Transformer. The structure made of DINOv2 and linear classifier takes the lead on popular few-shot segmentation bench mark PASCAL-$5^i$, substantially outperforming the best of ResNet structure by 116% in one-shot scenario. We demonstrate the great potential of large pretrained ViT-based model on GFSS task, and expect further improvement on testing benchmarks. However, a potential caveat is that when applying pure ViT-based model and large scale ViT decoder, the model is easy to overfit.

View on arXivView PDF

Code Implementations(1)

lygeng0427/ViTSegCommunity100%
MIT
60Shell, PythonDec 2, 20232 years agoMIT

Similar Papers

A Surprisingly Simple Approach to Generalized Few-Shot Semantic Segmentation

90% match2026

Memory-guided Network with Uncertainty-based Feature Augmentation for Few-shot Semantic Segmentation

88% match2024

Label Anything: Multi-Class Few-Shot Semantic Segmentation with Visual Prompts

87% match1 repo872024

FS-SAM2: Adapting Segment Anything Model 2 for Few-Shot Semantic Segmentation via Low-Rank Adaptation

87% match1 repo262025

Visual Prompting for Generalized Few-shot Segmentation: A Multi-scale Approach

87% match1 repo292026

High-Performance Few-Shot Segmentation with Foundation Models: An Empirical Study

87% match1 repo82024

Cite this paper

@article{geng2024applying,
  title  = {Applying ViT in Generalized Few-shot Semantic Segmentation},
  author = {Liyuan Geng and Jinhong Xia and Yuanhe Guo},
  year   = {2024},
  eprint = {2408.14957},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2408.14957v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2408.14957.svg)](https://codeofpaper.com/paper/2408.14957)

Discussion