Category Feature Transformer for Semantic Segmentation

Quan Tang, Chuanjian Liu, Fagui Liu, Yifan Liu, Jun Jiang +3 more
8/10/2023
cs.CV

No official code found. Best implementation: BebDong/EMOSeg (2 stars, last commit 2025-02-06).

Abstract

Aggregation of multi-stage features has been revealed to play a significant role in semantic segmentation. Unlike previous methods employing point-wise summation or concatenation for feature aggregation, this study proposes the Category Feature Transformer (CFT) that explores the flow of category embedding and transformation among multi-stage features through the prevalent multi-head attention mechanism. CFT learns unified feature embeddings for individual semantic categories from high-level features during each aggregation process and dynamically broadcasts them to high-resolution features. Integrating the proposed CFT into a typical feature pyramid structure exhibits superior performance over a broad range of backbone networks. We conduct extensive experiments on popular semantic segmentation benchmarks. Specifically, the proposed CFT obtains a compelling 55.1% mIoU with greatly reduced model parameters and computations on the challenging ADE20K dataset.

View on arXivView PDF

Code Implementations(1)

BebDong/EMOSegCommunity100%
20Shell, PythonOct 11, 20231 years ago

Similar Papers

A Decoding Scheme with Successive Aggregation of Multi-Level Features for Light-Weight Semantic Segmentation

85% match1 repo12024

CEDNet: A Cascade Encoder-Decoder Network for Dense Prediction

84% match1 repo1362023

Bilateral Network with Residual U-blocks and Dual-Guided Attention for Real-time Semantic Segmentation

84% match1 repo112023

Retro-FPN: Retrospective Feature Pyramid Network for Point Cloud Semantic Segmentation

84% match1 repo302023

Building Vision Transformers with Hierarchy Aware Feature Aggregation

84% match1 repo222023

Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Experts

84% match2024

Cite this paper

@article{tang2023category,
  title  = {Category Feature Transformer for Semantic Segmentation},
  author = {Quan Tang and Chuanjian Liu and Fagui Liu and Yifan Liu and Jun Jiang and Bowen Zhang and Kai Han and Yunhe Wang},
  year   = {2023},
  eprint = {2308.05581},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2308.05581v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2308.05581.svg)](https://codeofpaper.com/paper/2308.05581)

Discussion