JIST: Joint Image and Sequence Training for Sequential Visual Place Recognition

Gabriele Berton, Gabriele Trivigno, Barbara Caputo, Carlo Masone
3/28/2024
cs.CV

Official code: ga1i13o/JIST (24 stars, MIT, last commit 2023-12-15).

Abstract

Visual Place Recognition aims at recognizing previously visited places by relying on visual clues, and it is used in robotics applications for SLAM and localization. Since typically a mobile robot has access to a continuous stream of frames, this task is naturally cast as a sequence-to-sequence localization problem. Nevertheless, obtaining sequences of labelled data is much more expensive than collecting isolated images, which can be done in an automated way with little supervision. As a mitigation to this problem, we propose a novel Joint Image and Sequence Training protocol (JIST) that leverages large uncurated sets of images through a multi-task learning framework. With JIST we also introduce SeqGeM, an aggregation layer that revisits the popular GeM pooling to produce a single robust and compact embedding from a sequence of single-frame embeddings. We show that our model is able to outperform previous state of the art while being faster, using 8 times smaller descriptors, having a lighter architecture and allowing to process sequences of various lengths. Code is available at https://github.com/ga1i13o/JIST

View on arXivView PDF

Code Implementations(1)

ga1i13o/JISTOfficial100%
MIT

Official repository of the paper "JIST: Joint Image and Sequence Training for Sequential Visual Place Recognition"

241PythonDec 7, 20232 years agoMIT

Similar Papers

Towards an Accurate and Effective Robot Vision (The Problem of Topological Localization for Mobile Robots)

84% match2025

ProGEO: Generating Prompts through Image-Text Contrastive Learning for Visual Geo-localization

83% match1 repo502024

TextInPlace: Indoor Visual Place Recognition in Repetitive Structures with Scene Text Spotting and Verification

82% match1 repo182025

OFVL-MS: Once for Visual Localization across Multiple Indoor Scenes

82% match1 repo132026

BOX3D: Lightweight Camera-LiDAR Fusion for 3D Object Detection and Localization

82% match2024

STAMICS: Splat, Track And Map with Integrated Consistency and Semantics for Dense RGB-D SLAM

81% match2025

Cite this paper

@article{berton2024jist,
  title  = {JIST: Joint Image and Sequence Training for Sequential Visual Place Recognition},
  author = {Gabriele Berton and Gabriele Trivigno and Barbara Caputo and Carlo Masone},
  year   = {2024},
  eprint = {2403.19787},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.19787v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.19787.svg)](https://codeofpaper.com/paper/2403.19787)

Discussion