DeMansia: Mamba Never Forgets Any Tokens

Ricky Fang
8/4/2024
cs.CVcs.AI

Official code: catalpaaa/DeMansia (8 stars, GPL-3.0, last commit 2025-04-08).

Abstract

This paper examines the mathematical foundations of transformer architectures, highlighting their limitations particularly in handling long sequences. We explore prerequisite models such as Mamba, Vision Mamba (ViM), and LV-ViT that pave the way for our proposed architecture, DeMansia. DeMansia integrates state space models with token labeling techniques to enhance performance in image classification tasks, efficiently addressing the computational challenges posed by traditional transformers. The architecture, benchmark, and comparisons with contemporary models demonstrate DeMansia's effectiveness. The implementation of this paper is available on GitHub at https://github.com/catalpaaa/DeMansia

View on arXivView PDF

Code Implementations(1)

catalpaaa/DeMansiaOfficial100%
GPL-3.0
81C, C++Jun 2, 20241 years agoGPL-3.0

Similar Papers

Scaling White-Box Transformers for Vision

84% match1 repo482024

Efficient and Accurate Scene Text Recognition with Cascaded-Transformers

83% match2025

Snakes and Ladders: Two Steps Up for VideoMamba

82% match1 repo482024

Introduction to Sequence Modeling with Transformers

82% match1 repo2025

Scaling White-Box Transformers for Vision

82% match1 repo482026

Segformer++: Efficient Token-Merging Strategies for High-Resolution Semantic Segmentation

82% match1 repo292024

Cite this paper

@article{fang2024demansia,
  title  = {DeMansia: Mamba Never Forgets Any Tokens},
  author = {Ricky Fang},
  year   = {2024},
  eprint = {2408.01986},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2408.01986v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2408.01986.svg)](https://codeofpaper.com/paper/2408.01986)

Discussion