TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation

Wenqiang Zhang, Zilong Huang, Guozhong Luo, Tao Chen, Xinggang Wang +3 more
4/12/2022
cs.CV

Official code: hustvl/TopFormer (406 stars, last commit 2022-10-27).

Abstract

Although vision transformers (ViTs) have achieved great success in computer vision, the heavy computational cost hampers their applications to dense prediction tasks such as semantic segmentation on mobile devices. In this paper, we present a mobile-friendly architecture named \textbf{To}ken \textbf{P}yramid Vision Trans\textbf{former} (\textbf{TopFormer}). The proposed \textbf{TopFormer} takes Tokens from various scales as input to produce scale-aware semantic features, which are then injected into the corresponding tokens to augment the representation. Experimental results demonstrate that our method significantly outperforms CNN- and ViT-based networks across several semantic segmentation datasets and achieves a good trade-off between accuracy and latency. On the ADE20K dataset, TopFormer achieves 5\% higher accuracy in mIoU than MobileNetV3 with lower latency on an ARM-based mobile device. Furthermore, the tiny version of TopFormer achieves real-time inference on an ARM-based mobile device with competitive results. The code and models are available at: https://github.com/hustvl/TopFormer

View on arXivView PDF

Code Implementations(1)

hustvl/TopFormerOfficial100%

TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation, CVPR2022

40643Shell, PythonMar 23, 20223 years agoNOASSERTION
mobile-semantic-segmentationsemantic-segmentation

Similar Papers

PP-MobileSeg: Explore the Fast and Accurate Semantic Segmentation Model on Mobile Devices

90% match1 repo9.4k2023

Dynamic Token Pruning in Plain Vision Transformers for Semantic Segmentation

86% match2023

P2AT: Pyramid Pooling Axial Transformer for Real-time Semantic Segmentation

85% match2023

Real-Time Semantic Segmentation on FPGA for Autonomous Vehicles Using LMIINet with the CGRA4ML Framework

84% match2025

CENet: Toward Concise and Efficient LiDAR Semantic Segmentation for Autonomous Driving

84% match1 repo1202022

Dynamic Spectrum Mixer for Visual Recognition

83% match2023

Cite this paper

@article{zhang2022topformer,
  title  = {TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation},
  author = {Wenqiang Zhang and Zilong Huang and Guozhong Luo and Tao Chen and Xinggang Wang and Wenyu Liu and Gang Yu and Chunhua Shen},
  year   = {2022},
  eprint = {2204.05525},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2204.05525}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2204.05525.svg)](https://codeofpaper.com/paper/2204.05525)

Discussion