Spectral Vision Transformer for Efficient Tokenization with Limited Data

Alexandra G. Roberts, Maneesh John, Jinwei Zhang, Dominick Romano, Mert Sisman +8 more
5/12/2026
cs.CVcs.AIeess.SP

No official code found. Best implementation: agr78/SpectralViT (3 stars, MIT, last commit 2026-05-13).

Abstract

We propose a novel spectral vision transformer architecture for efficient tokenization in limited data, with an emphasis on medical imaging. We outline convenient theoretical properties arising from the choice of basis including spatial invariance and optimal signal-to-noise ratio. We show reduced complexity arising from the spectral projection compared to spatial vision transformers. We show equitable or superior performance with a reduced number of parameters as compared to a variety of models including compact and standard vision transformers, convolutional neural networks with attention, shifted window transformers, multi-layer perceptrons, and logistic regression. We include simulated, public, and clinical data in our analysis and release our code at: \verb+github.com/agr78/spectralViT+.

View on arXivView PDF

Code Implementations(1)

agr78/SpectralViTCommunity75%
MIT

Spectral vision transformer for efficient tokenization with limited data

30Feb 13, 20265 months agoMIT
classificationcomputer-visioncornellcornell-universitydata-scarce-environments+13 more

Similar Papers

A Comprehensive Survey of Mamba Architectures for Medical Image Analysis: Classification, Segmentation, Restoration and Beyond

86% match1 repo752024

EfficientMorph: Parameter-Efficient Transformer-Based Architecture for 3D Image Registration

86% match2024

Large-kernel Attention for Efficient and Robust Brain Lesion Segmentation

86% match1 repo62023

Pretrained ViTs Yield Versatile Representations For Medical Images

86% match1 repo1132023

Vision Transformer for Efficient Chest X-ray and Gastrointestinal Image Classification

85% match2023

U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation

85% match1 repo5612024

Cite this paper

@article{roberts2026spectral,
  title  = {Spectral Vision Transformer for Efficient Tokenization with Limited Data},
  author = {Alexandra G. Roberts and Maneesh John and Jinwei Zhang and Dominick Romano and Mert Sisman and Ki Sueng Choi and Heejong Kim and Mert R. Sabuncu and Thanh D. Nguyen and Alexey V. Dimov and Pascal Spincemaille and Brian H. Kopell and Yi Wang},
  year   = {2026},
  eprint = {2605.12026},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2605.12026v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2605.12026.svg)](https://codeofpaper.com/paper/2605.12026)

Discussion