Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image

Fangchang Ma, Sertac Karaman
9/21/2017
cs.ROcs.AIcs.CV

Official code: fangchangma/sparse-to-dense (447 stars, last commit 2018-07-21).

Abstract

We consider the problem of dense depth prediction from a sparse set of depth measurements and a single RGB image. Since depth estimation from monocular images alone is inherently ambiguous and unreliable, to attain a higher level of robustness and accuracy, we introduce additional sparse depth samples, which are either acquired with a low-resolution depth sensor or computed via visual Simultaneous Localization and Mapping (SLAM) algorithms. We propose the use of a single deep regression network to learn directly from the RGB-D raw data, and explore the impact of number of depth samples on prediction accuracy. Our experiments show that, compared to using only RGB images, the addition of 100 spatially random depth samples reduces the prediction root-mean-square error by 50% on the NYU-Depth-v2 indoor dataset. It also boosts the percentage of reliable prediction from 59% to 92% on the KITTI dataset. We demonstrate two applications of the proposed algorithm: a plug-in module in SLAM to convert sparse maps to dense maps, and super-resolution for LiDARs. Software and video demonstration are publicly available.

View on arXivView PDF

Code Implementations(1)

fangchangma/sparse-to-denseOfficial100%

ICRA 2018 "Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image" (Torch Implementation)

44794LuaSep 12, 20178 years agoNOASSERTION
computer-visiondeep-learningdepth-completiondepth-estimationdepth-image+3 more

Similar Papers

StarryGazer: Leveraging Monocular Depth Estimation Models for Domain-Agnostic Single Depth Image Completion

89% match2025

RadarCam-Depth: Radar-Camera Fusion for Depth Estimation with Learned Metric Scale

88% match1 repo682024

Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera

88% match1 repo6542018

Semi-Supervised Monocular Depth Estimation with Left-Right Consistency Using Deep Neural Network

88% match1 repo922019

RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion

87% match1 repo272023

Deep Ordinal Regression Network for Monocular Depth Estimation

87% match1 repo4712018

Cite this paper

@article{ma2017sparsetodense,
  title  = {Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image},
  author = {Fangchang Ma and Sertac Karaman},
  year   = {2017},
  eprint = {1709.07492},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1709.07492}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1709.07492.svg)](https://codeofpaper.com/paper/1709.07492)

Discussion