Self-Supervised Video Similarity Learning

Giorgos Kordopatis-Zilos, Giorgos Tolias, Christos Tzelepis, Ioannis Kompatsiaris, Ioannis Patras +1 more
4/6/2023
cs.CVcs.LG

Official code: gkordo/s2vs (44 stars, MIT, last commit 2023-11-25).

Abstract

We introduce S$^2$VS, a video similarity learning approach with self-supervision. Self-Supervised Learning (SSL) is typically used to train deep models on a proxy task so as to have strong transferability on target tasks after fine-tuning. Here, in contrast to prior work, SSL is used to perform video similarity learning and address multiple retrieval and detection tasks at once with no use of labeled data. This is achieved by learning via instance-discrimination with task-tailored augmentations and the widely used InfoNCE loss together with an additional loss operating jointly on self-similarity and hard-negative similarity. We benchmark our method on tasks where video relevance is defined with varying granularity, ranging from video copies to videos depicting the same incident or event. We learn a single universal model that achieves state-of-the-art performance on all tasks, surpassing previously proposed methods that use labeled data. The code and pretrained models are publicly available at: https://github.com/gkordo/s2vs

View on arXivView PDF

Code Implementations(1)

gkordo/s2vsOfficial100%
MIT

Authors official PyTorch implementation of the "Self-Supervised Video Similarity Learning" [CVPRW 2023]

443Shell, PythonApr 5, 20232 years agoMIT
duplicate-videosfivrndvrself-supervised-learningself-supervision+6 more

Similar Papers

Learning the Relation between Similarity Loss and Clustering Loss in Self-Supervised Learning

81% match1 repo502023

A Large-Scale Analysis on Contextual Self-Supervised Video Representation Learning

80% match2025

SEVERE++: Evaluating Benchmark Sensitivity in Generalization of Video Representation Learning

80% match2025

3D sans 3D Scans: Scalable Pre-training from Video-Generated Point Clouds

80% match2025

When the Future Becomes the Past: Taming Temporal Correspondence for Self-supervised Video Representation Learning

80% match1 repo192026

Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representations

80% match1 repo312026

Cite this paper

@article{kordopatiszilos2023selfsupervised,
  title  = {Self-Supervised Video Similarity Learning},
  author = {Giorgos Kordopatis-Zilos and Giorgos Tolias and Christos Tzelepis and Ioannis Kompatsiaris and Ioannis Patras and Symeon Papadopoulos},
  year   = {2023},
  eprint = {2304.03378},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2304.03378v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.03378.svg)](https://codeofpaper.com/paper/2304.03378)

Discussion