PromptLA: Towards Integrity Verification of Black-box Text-to-Image Diffusion Models

Zhuomeng Zhang, Fangqi Li, Chong Di, Hongyu Zhu, Hanyi Wang +1 more
12/20/2024
cs.CVcs.AIcs.CR

No official code found. Best implementation: zzmsmm/PromptLA (1 star, last commit 2024-11-06).

Abstract

Despite the impressive synthesis quality of text-to-image (T2I) diffusion models, their black-box deployment poses significant regulatory challenges: Malicious actors can fine-tune these models to generate illegal content, circumventing existing safeguards through parameter manipulation. Therefore, it is essential to verify the integrity of T2I diffusion models. To this end, considering the randomness within the outputs of generative models and the high costs in interacting with them, we discern model tampering via the KL divergence between the distributions of the features of generated images. We propose a novel prompt selection algorithm based on learning automaton (PromptLA) for efficient and accurate verification. Evaluations on four advanced T2I models (e.g., SDXL, FLUX.1) demonstrate that our method achieves a mean AUC of over 0.96 in integrity detection, exceeding baselines by more than 0.2, showcasing strong effectiveness and generalization. Additionally, our approach achieves lower cost and is robust against image-level post-processing. To the best of our knowledge, this paper is the first work addressing the integrity verification of T2I diffusion models, which establishes quantifiable standards for AI copyright litigation in practice.

View on arXivView PDF

Code Implementations(1)

zzmsmm/PromptLACommunity60%

This repo is implementation for PromptLA: Towards Integrity Verification of Black-box Text-to-Image Diffusion Models

10Shell, PythonAug 16, 20241 years ago

Similar Papers

Cite this paper

@article{zhang2024promptla,
  title  = {PromptLA: Towards Integrity Verification of Black-box Text-to-Image Diffusion Models},
  author = {Zhuomeng Zhang and Fangqi Li and Chong Di and Hongyu Zhu and Hanyi Wang and Shilin Wang},
  year   = {2024},
  eprint = {2412.16257},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.16257v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.16257.svg)](https://codeofpaper.com/paper/2412.16257)

Discussion