LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry

Abstract

Visual odometry estimates the motion of a moving cam-era based on visual input. Existing methods, mostly focusing on two-view point tracking, often ignore the rich tempo-ral context in the image sequence, thereby overlooking the global motion patterns and providing no assessment of the full trajectory reliability. These shortcomings hinder per-formance in scenarios with occlusion, dynamic objects, and low-texture areas. To address these challenges, we present the Long-term Effective Any Point Tracking (LEAP) mod-ule. LEAP innovatively combines visual, inter-track, and temporal cues with mindfully selected anchors for dynamic track estimation. Moreover, LEAP's temporal probabilistic formulation integrates distribution updates into a learnable iterative refinement module to reason about point-wise un-certainty. Based on these traits, we develop LEAP-VO, a robust visual odometry system adept at handling occlusions and dynamic scenes. Our mindful integration showcases a novel practice by employing long-term point tracking as the front-end. Extensive experiments demonstrate that the pro-posed pipeline significantly outperforms existing baselines across various visual odometry benchmarks.