Reconstructing Animatable Categories from Videos

Gengshan Yang, Chaoyang Wang, Dinesh Reddy Narapureddy, Deva Ramanan
5/1/2023

No official code found. Best implementation: lab4d-org/lab4d (354 stars, MIT, last commit 2025-02-02). 1 more below.

Abstract

Building animatable 3D models is challenging due to the need for 3D scans, laborious registration, and rigging. Recently, differentiable rendering provides a pathway to obtain high-quality 3D models from monocular videos, but these are limited to rigid categories or single instances. We present RAC, a method to build category-level 3D models from monocular videos, disentangling variations over instances and motion over time. Three key ideas are introduced to solve this problem: (1) specializing a category-level skeleton to instances, (2) a method for latent space regularization that encourages shared structure across a category while maintaining instance details, and (3) using 3D background models to disentangle objects from the background. We build 3D models for humans, cats and dogs given monocular videos. Project page: https://gengshan-y.github.io/rac-www/.

View on arXivView PDF

Code Implementations(2)

lab4d-org/lab4dCommunity100%
MIT

A framework for 4D reconstruction from monocular videos.

35425HTML, ShellMay 7, 20231 years agoMIT
4d-reconstructiondifferentiable-renderingvideo-reconstruction
gengshan-y/racCommunity100%

RAC: Reconstructing Animatable Categories from Videos. CVPR 2023

752C, C++Jan 5, 20233 years ago

Similar Papers

Virtual Pets: Animatable Animal Generation in 3D Scenes

83% match2023

FMGS-Avatar: Mesh-Guided 2D Gaussian Splatting with Foundation Model Priors for 3D Monocular Avatar Reconstruction

83% match2025

Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos

83% match1 repo862024

PPR: Physically Plausible Reconstruction from Monocular Videos

83% match2 repos822023

MagicPose4D: Crafting Articulated Models with Appearance and Motion Control

82% match1 repo1122024

AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation

82% match2023

Cite this paper

@article{yang2023reconstructing,
  title  = {Reconstructing Animatable Categories from Videos},
  author = {Gengshan Yang and Chaoyang Wang and Dinesh Reddy Narapureddy and Deva Ramanan},
  year   = {2023},
  eprint = {2305.06351},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/pdf/2305.06351},
  journal = {CVPR 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.06351.svg)](https://codeofpaper.com/paper/2305.06351)

Discussion