Adapting Vision Transformers to Ultra-High Resolution Semantic Segmentation with Relay Tokens

Yohann Perron, Vladyslav Sydorov, Christophe Pottier, Loic Landrieu
1/9/2026
cs.CV

No official code found. Best implementation: YohannPerron/RelayToken (7 stars, MIT, last commit 2026-01-23).

Abstract

Current approaches for segmenting ultra high resolution images either slide a window, thereby discarding global context, or downsample and lose fine detail. We propose a simple yet effective method that brings explicit multi scale reasoning to vision transformers, simultaneously preserving local details and global awareness. Concretely, we process each image in parallel at a local scale (high resolution, small crops) and a global scale (low resolution, large crops), and aggregate and propagate features between the two branches with a small set of learnable relay tokens. The design plugs directly into standard transformer backbones (eg ViT and Swin) and adds fewer than 2 % parameters. Extensive experiments on three ultra high resolution segmentation benchmarks, Archaeoscape, URUR, and Gleason, and on the conventional Cityscapes dataset show consistent gains, with up to 15 % relative mIoU improvement. Code and pretrained models are available at https://archaeoscape.ai/work/relay-tokens/ .

View on arXivView PDF

Code Implementations(1)

YohannPerron/RelayTokenCommunity74%
MIT

Adapting Vision Transformers to Ultra-High Resolution Semantic Segmentation with Relay Tokens

70Python, Jupyter NotebookJan 9, 20269 months agoMIT

Similar Papers

ULTra: Unveiling Latent Token Interpretability in Transformer-Based Understanding and Segmentation

85% match2024

Ultra-High Resolution Segmentation via Boundary-Enhanced Patch-Merging Transformer

85% match2024

Context-Aware Semantic Segmentation via Stage-Wise Attention

85% match2026

A Decoding Scheme with Successive Aggregation of Multi-Level Features for Light-Weight Semantic Segmentation

84% match1 repo12024

Progressive Token Length Scaling in Transformer Encoders for Efficient Universal Segmentation

84% match1 repo72026

Leveraging Adaptive Implicit Representation Mapping for Ultra High-Resolution Image Segmentation

84% match2024

Cite this paper

@article{perron2026adapting,
  title  = {Adapting Vision Transformers to Ultra-High Resolution Semantic Segmentation with Relay Tokens},
  author = {Yohann Perron and Vladyslav Sydorov and Christophe Pottier and Loic Landrieu},
  year   = {2026},
  eprint = {2601.05927},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2601.05927v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2601.05927.svg)](https://codeofpaper.com/paper/2601.05927)

Discussion