Self-supervised Learning of Contextualized Local Visual Embeddings

Thalles Santos Silva, Helio Pedrini, Adín Ramírez Rivera
10/1/2023
cs.CV

No official code found. Best implementation: sthalles/CLoVE (8 stars, MIT, last commit 2023-12-26).

Abstract

We present Contextualized Local Visual Embeddings (CLoVE), a self-supervised convolutional-based method that learns representations suited for dense prediction tasks. CLoVE deviates from current methods and optimizes a single loss function that operates at the level of contextualized local embeddings learned from output feature maps of convolution neural network (CNN) encoders. To learn contextualized embeddings, CLoVE proposes a normalized mult-head self-attention layer that combines local features from different parts of an image based on similarity. We extensively benchmark CLoVE's pre-trained representations on multiple datasets. CLoVE reaches state-of-the-art performance for CNN-based architectures in 4 dense prediction downstream tasks, including object detection, instance segmentation, keypoint detection, and dense pose estimation.

View on arXivView PDF

Code Implementations(1)

sthalles/CLoVECommunity100%
MIT

Official PyTroch implementation for Self-supervised Learning of Contextualized Local Visual Embeddings (CLoVE)

81Shell, PythonAug 17, 20232 years agoMIT

Similar Papers

Multi-Level Contrastive Learning for Dense Prediction Task

87% match1 repo62023

SODAWideNet++: Combining Attention and Convolutions for Salient Object Detection

85% match1 repo12024

Patch-Level Kernel Alignment for Dense Self-Supervised Learning

85% match2025

CNN-JEPA: Self-Supervised Pretraining Convolutional Neural Networks Using Joint Embedding Predictive Architecture

85% match1 repo382024

DenseDINO: Boosting Dense Self-Supervised Learning with Token-Based Point-Level Consistency

85% match2023

Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions

84% match1 repo1.9k2021

Cite this paper

@article{silva2023selfsupervised,
  title  = {Self-supervised Learning of Contextualized Local Visual Embeddings},
  author = {Thalles Santos Silva and Helio Pedrini and Adín Ramírez Rivera},
  year   = {2023},
  eprint = {2310.00527},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.00527v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.00527.svg)](https://codeofpaper.com/paper/2310.00527)

Discussion