Rethinking the Correlation in Few-Shot Segmentation: A Buoys View

Abstract

Few-shot segmentation (FSS) aims to segment novel ob-jects in a given query image with only a few annotated support images. However, most previous best-performing methods, whether prototypical learning methods or affinity learning methods, neglect to alleviate false matches caused by their own pixel-level correlation. In this work, we rethink how to mitigate the false matches from the perspective of representative reference features (referred to as buoys), and propose a novel adaptive buoys correlation (ABC) network to rectify direct pairwise pixel-level correlation, including a buoys mining module and an adaptive correlation module. The proposed ABC enjoys several merits. First, to learn the buoys well without any correspondence supervision, we customize the buoys mining module according to the three characteristics of representativeness, task awareness and re-silience. Second, the proposed adaptive correlation module is responsible for further endowing buoy-correlation-based pixel matching with an adaptive ability. Extensive experimen-tal results with two different backbones on two challenging benchmarks demonstrate that our ABC, as a general plu-gin, achieves consistent improvements over several leading methods on both I-shot and 5-shot settings.