AFIDAF: Alternating Fourier and Image Domain Adaptive Filters as an Efficient Alternative to Attention in ViTs

Yunling Zheng, Zeyi Xu, Fanghui Xue, Biao Yang, Jiancheng Lyu +3 more
7/16/2024
cs.CV

No confident code implementation found for this paper.

Abstract

We propose and demonstrate an alternating Fourier and image domain filtering approach for feature extraction as an efficient alternative to build a vision backbone without using the computationally intensive attention. The performance among the lightweight models reaches the state-of-the-art level on ImageNet-1K classification, and improves downstream tasks on object detection and segmentation consistently as well. Our approach also serves as a new tool to compress vision transformers (ViTs).

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

LAM3D: Leveraging Attention for Monocular 3D Object Detection

85% match2024

Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Model

85% match1 repo1.0k2024

FViT: A Focal Vision Transformer with Gabor Filter

84% match1 repo192024

ReViT: Enhancing Vision Transformers Feature Diversity with Attention Residual Connections

84% match1 repo312024

APLA: A Simple Adaptation Method for Vision Transformers

84% match1 repo162025

Skip-Attention: Improving Vision Transformers by Paying Less Attention

83% match2026

Cite this paper

@article{zheng2024afidaf,
  title  = {AFIDAF: Alternating Fourier and Image Domain Adaptive Filters as an Efficient Alternative to Attention in ViTs},
  author = {Yunling Zheng and Zeyi Xu and Fanghui Xue and Biao Yang and Jiancheng Lyu and Shuai Zhang and Yingyong Qi and Jack Xin},
  year   = {2024},
  eprint = {2407.12217},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.12217v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.12217.svg)](https://codeofpaper.com/paper/2407.12217)

Discussion