PatchFusion: An End-to-End Tile-Based Framework for High-Resolution Monocular Metric Depth Estimation

Abstract

Single image depth estimation is a foundational task in computer vision and generative modeling. However, prevailing depth estimation models grapple with accom-modating the increasing resolutions commonplace in to-day's consumer cameras and devices. Existing high-resolution strategies show promise, but they often face limi-tations, ranging from error propagation to the loss of high-frequency details. We present PatchFusion, a novel tile-based framework with three key components to improve the current state of the art: (1) A patch-wise fusion network that fuses a globally-consistent coarse prediction with finer, in-consistent tiled predictions via high-level feature guidance, (2) A Global-to-Local (G2L) module that adds vital con-text to the fusion network, discarding the need for patch selection heuristics, and (3) A Consistency-Aware Training (CAT) and Inference (CAI) approach, emphasizing patch overlap consistency and thereby eradicating the neces-sity for post-processing. Experiments on Unreal. Stereo 4K, MVS-Synth, and Middleburry 2014 demonstrate that our framework can generate high-resolution depth maps with intricate details. PatchFusion is independent of the base model for depth estimation. Notably, our framework built on top of SOTA ZoeDepth brings improvements for a total of 17.3% and 29.4% in terms of the root mean squared error (RMSE) on UnrealStereo4K and MVS-Synth, respectively.