Hierarchical Spatiotemporal Transformers for Video Object Segmentation

Jun-Sang Yoo, Hongjae Lee, Seung-Won Jung
7/17/2023
cs.CV

No confident code implementation found for this paper.

Abstract

This paper presents a novel framework called HST for semi-supervised video object segmentation (VOS). HST extracts image and video features using the latest Swin Transformer and Video Swin Transformer to inherit their inductive bias for the spatiotemporal locality, which is essential for temporally coherent VOS. To take full advantage of the image and video features, HST casts image and video features as a query and memory, respectively. By applying efficient memory read operations at multiple scales, HST produces hierarchical features for the precise reconstruction of object masks. HST shows effectiveness and robustness in handling challenging scenarios with occluded and fast-moving objects under cluttered backgrounds. In particular, HST-B outperforms the state-of-the-art competitors on multiple popular benchmarks, i.e., YouTube-VOS (85.0%), DAVIS 2017 (85.9%), and DAVIS 2016 (94.0%).

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Robust and Efficient Memory Network for Video Object Segmentation

92% match2023

Region Aware Video Object Segmentation with Deep Motion Modeling

91% match2022

READMem: Robust Embedding Association for a Diverse Memory in Unconstrained Video Object Segmentation

88% match1 repo62023

Memory-Efficient Continual Learning Object Segmentation for Long Video

88% match2023

Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentation

88% match1 repo1092023

XMem++: Production-level Video Segmentation From Few Annotated Frames

88% match1 repo2122023

Cite this paper

@article{yoo2023hierarchical,
  title  = {Hierarchical Spatiotemporal Transformers for Video Object Segmentation},
  author = {Jun-Sang Yoo and Hongjae Lee and Seung-Won Jung},
  year   = {2023},
  eprint = {2307.08263},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2307.08263v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2307.08263.svg)](https://codeofpaper.com/paper/2307.08263)

Discussion