Spike2Former: Efficient Spiking Transformer for High-performance Image Segmentation

Zhenxin Lei, Man Yao, Jiakui Hu, Xinhao Luo, Yanye Lu +2 more
12/19/2024
cs.CVcs.AIcs.NE

No official code found. Best implementation: BICLab/Spike2Former (44 stars, last commit 2026-01-14).

Abstract

Spiking Neural Networks (SNNs) have a low-power advantage but perform poorly in image segmentation tasks. The reason is that directly converting neural networks with complex architectural designs for segmentation tasks into spiking versions leads to performance degradation and non-convergence. To address this challenge, we first identify the modules in the architecture design that lead to the severe reduction in spike firing, make targeted improvements, and propose Spike2Former architecture. Second, we propose normalized integer spiking neurons to solve the training stability problem of SNNs with complex architectures. We set a new state-of-the-art for SNNs in various semantic segmentation datasets, with a significant improvement of +12.7% mIoU and 5.0 efficiency on ADE20K, +14.3% mIoU and 5.2 efficiency on VOC2012, and +9.1% mIoU and 6.6 efficiency on CityScapes.

View on arXivView PDF

Code Implementations(1)

BICLab/Spike2FormerCommunity100%

Offical implementation of "Spike2Former: Efficient Spiking Transformer for High-performance Image Segmentation" (AAAI2025 Oral)

449C++, CSSDec 10, 20249 months ago

Similar Papers

AT-SNN: Adaptive Tokens for Vision Transformer on Spiking Neural Network

85% match2024

Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detection

85% match1 repo2592026

EvSegSNN: Neuromorphic Semantic Segmentation for Event Data

84% match2024

SLTNet: Efficient Event-based Semantic Segmentation with Spike-driven Lightweight Transformer-based Networks

82% match1 repo42024

U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation

82% match1 repo5612024

Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips

82% match1 repo2352026

Cite this paper

@article{lei2024spikeformer,
  title  = {Spike2Former: Efficient Spiking Transformer for High-performance Image Segmentation},
  author = {Zhenxin Lei and Man Yao and Jiakui Hu and Xinhao Luo and Yanye Lu and Bo Xu and Guoqi Li},
  year   = {2024},
  eprint = {2412.14587},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.14587v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.14587.svg)](https://codeofpaper.com/paper/2412.14587)

Discussion