ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library

Mohammadreza Shetab-Bushehri, Miguel Aranda, Youcef Mezouar, Adrien Bartoli, Erol Ozgur
1/10/2023
cs.CV

Official code: mrshetab/ROBUSfT (33 stars, last commit 2023-06-26).

Abstract

Tracking the 3D shape of a deforming object using only monocular 2D vision is a challenging problem. This is because one should (i) infer the 3D shape from a 2D image, which is a severely underconstrained problem, and (ii) implement the whole solution pipeline in real-time. The pipeline typically requires feature detection and matching, mismatch filtering, 3D shape inference and feature tracking algorithms. We propose ROBUSfT, a conventional pipeline based on a template containing the object's rest shape, texturemap and deformation law. ROBUSfT is ready-to-use, wide-baseline, capable of handling large deformations, fast up to 30 fps, free of training, and robust against partial occlusions and discontinuity in video frames. It outperforms the state-of-the-art methods in challenging datasets. ROBUSfT is implemented as a publicly available C++ library and we provide a tutorial on how to use it in https://github.com/mrshetab/ROBUSfT

View on arXivView PDF

Code Implementations(1)

mrshetab/ROBUSfTOfficial100%

A C++ Library for Monocular Real-Time 3D Shape Tracking of Isometrically Deforming Objects

339C++, CMakeOct 26, 20213 years ago

Similar Papers

PAD3R: Pose-Aware Dynamic 3D Reconstruction from Casual Videos

84% match2025

Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scale

81% match2025

U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Clouds

81% match1 repo202023

SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Image

81% match1 repo272024

SEMAGIC: Learning Semantically Consistent Deformable 3D Representations from In-the-Wild Images

81% match2026

TFS-NeRF: Template-Free NeRF for Semantic 3D Reconstruction of Dynamic Scene

80% match1 repo2024

Cite this paper

@article{shetabbushehri2023robusft,
  title  = {ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library},
  author = {Mohammadreza Shetab-Bushehri and Miguel Aranda and Youcef Mezouar and Adrien Bartoli and Erol Ozgur},
  year   = {2023},
  eprint = {2301.04037},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.04037v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.04037.svg)](https://codeofpaper.com/paper/2301.04037)

Discussion