PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation

Xinqi Xiong, Andrea Dunn Beltran, Jun Myeong Choi, Marc Niethammer, Roni Sengupta
4/23/2025
cs.CV

Official code: anaxqx/PPS-Ctrl (9 stars, last commit 2025-06-19).

Abstract

Accurate depth estimation enhances endoscopy navigation and diagnostics, but obtaining ground-truth depth in clinical settings is challenging. Synthetic datasets are often used for training, yet the domain gap limits generalization to real data. We propose a novel image-to-image translation framework that preserves structure while generating realistic textures from clinical data. Our key innovation integrates Stable Diffusion with ControlNet, conditioned on a latent representation extracted from a Per-Pixel Shading (PPS) map. PPS captures surface lighting effects, providing a stronger structural constraint than depth maps. Experiments show our approach produces more realistic translations and improves depth estimation over GAN-based MI-CycleGAN. Our code is publicly accessible at https://github.com/anaxqx/PPS-Ctrl.

View on arXivView PDF

Code Implementations(1)

anaxqx/PPS-CtrlOfficial100%
91PythonMar 16, 20251 years ago

Similar Papers

Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos

88% match1 repo452026

Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videos

88% match1 repo452024

EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Camera

86% match1 repo972024

Structure-preserving Image Translation for Depth Estimation in Colonoscopy Video

86% match1 repo102024

EndoOmni: Zero-Shot Cross-Dataset Depth Estimation in Endoscopy by Robust Self-Learning from Noisy Labels

85% match1 repo212024

Self-supervised Monocular Depth and Pose Estimation for Endoscopy with Generative Latent Priors

85% match2024

Cite this paper

@article{xiong2025ppsctrl,
  title  = {PPS-Ctrl: Controllable Sim-to-Real Translation for Colonoscopy Depth Estimation},
  author = {Xinqi Xiong and Andrea Dunn Beltran and Jun Myeong Choi and Marc Niethammer and Roni Sengupta},
  year   = {2025},
  eprint = {2504.17067},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2504.17067v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2504.17067.svg)](https://codeofpaper.com/paper/2504.17067)

Discussion