Cosalpure: Learning Concept from Group Images for Robust Co-Saliency Detection

Abstract

Co-salient object detection (CoSOD) aims to identify the common and salient (usually in the foreground) regions across a given group of images. Although achieving sig-nificant progress, state-of-the-art CoSODs could be easily affected by some adversarial perturbations, leading to sub-stantial accuracy reduction. The adversarial perturbations can mislead CoSODs but do not change the high-level se-mantic information (e.g., concept) of the co-salient objects. In this paper, we propose a novel robustness enhancement framework by first learning the concept of the co-salient ob-jects based on the input group images and then leveraging this concept to purify adversarial perturbations, which are subsequently fed to CoSODs for robustness enhancement. Specifically, we propose Cosalpure containing two modules, i.e., group-image concept learning and concept-guided diffusion purification. For the first module, we adopt a pre-trained text-to-image diffusion model to learn the con-cept of co-salient objects within group images where the learned concept is robust to adversarial examples. For the second module, we map the adversarial image to the latent space and then perform diffusion generation by embedding the learned concept into the noise prediction function as an extra condition. Our method can effectively alleviate the in-fluence of the SOTA adversarial attack containing different adversarial patterns, including exposure and noise. The ex-tensive results demonstrate that our method could enhance the robustness of Cos ODs significantly. The project is avail-able at https://vllen.github.io/CosalPure/.