OmniGlue: Generalizable Feature Matching with Foundation Model Guidance

Abstract

The image matching field has been witnessing a contin-uous emergence of novel learnable feature matching techniques, with ever-improving performance on conventional benchmarks. However, our investigation shows that de-spite these gains, their potential for real-world applications is restricted by their limited generalization capabili-ties to novel image domains. In this paper, we introduce OmniGlue, the first learnable image matcher that is de-signed with generalization as a core principle. OmniGlue leverages broad knowledge from a vision foundation model to guide the feature matching process, boosting general-ization to domains not seen at training time. Addition-ally, we propose a novel keypoint position-guided attention mechanism which disentangles spatial and appear-ance information, leading to enhanced matching descrip-tors. We perform comprehensive experiments on a suite of 7 datasets with varied image domains, including scene-level, object-centric and aerial images. OmniGlue's novel components lead to relative gains on unseen domains of 20.9% with respect to a directly comparable reference model, while also outperforming the recent LightGlue method by 9.5% relatively. Code and model can be found at https://hwjiang151o.github.io/OmniGlue.