MVTrans: Multi-View Perception of Transparent Objects

Yi Ru Wang, Yuchi Zhao, Haoping Xu, Saggi Eppel, Alan Aspuru-Guzik +2 more
2/22/2023
cs.ROcs.AIcs.CV

No official code found. Best implementation: ac-rad/MVTrans (20 stars, last commit 2023-08-08).

Abstract

Transparent object perception is a crucial skill for applications such as robot manipulation in household and laboratory settings. Existing methods utilize RGB-D or stereo inputs to handle a subset of perception tasks including depth and pose estimation. However, transparent object perception remains to be an open problem. In this paper, we forgo the unreliable depth map from RGB-D sensors and extend the stereo based method. Our proposed method, MVTrans, is an end-to-end multi-view architecture with multiple perception capabilities, including depth estimation, segmentation, and pose estimation. Additionally, we establish a novel procedural photo-realistic dataset generation pipeline and create a large-scale transparent object detection dataset, Syn-TODD, which is suitable for training networks with all three modalities, RGB-D, stereo and multi-view RGB. Project Site: https://ac-rad.github.io/MVTrans/

View on arXivView PDF

Code Implementations(1)

ac-rad/MVTransCommunity72%

MVTrans: Multi-view Perception to See Transparent Objects (ICRA2023)

203HTML, ShellSep 15, 20223 years ago

Similar Papers

Monocular Depth Estimation and Segmentation for Transparent Object with Iterative Semantic and Geometric Fusion

89% match1 repo292025

ClearDepth: Enhanced Stereo Perception of Transparent Objects for Robotic Manipulation

88% match2024

TransNet: Transparent Object Manipulation Through Category-Level Pose Estimation

87% match2023

ASGrasp: Generalizable Transparent Object Reconstruction and 6-DoF Grasp Detection from RGB-D Active Stereo Camera

86% match1 repo1012024

Transparent Object Depth Completion

86% match1 repo1222024

Category-level Object Detection, Pose Estimation and Reconstruction from Stereo Images

85% match1 repo22024

Cite this paper

@article{wang2023mvtrans,
  title  = {MVTrans: Multi-View Perception of Transparent Objects},
  author = {Yi Ru Wang and Yuchi Zhao and Haoping Xu and Saggi Eppel and Alan Aspuru-Guzik and Florian Shkurti and Animesh Garg},
  year   = {2023},
  eprint = {2302.11683},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2302.11683v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.11683.svg)](https://codeofpaper.com/paper/2302.11683)

Discussion