Disentangling Mean Embeddings for Better Diagnostics of Image Generators

Sebastian G. Gruber, Pascal Tobias Ziegler, Florian Buettner
9/2/2024
cs.CVcs.LG

No official code found. Best implementation: MLO-lab/Disentangling_Mean_Embeddings (1 star, last commit 2024-12-28).

Abstract

The evaluation of image generators remains a challenge due to the limitations of traditional metrics in providing nuanced insights into specific image regions. This is a critical problem as not all regions of an image may be learned with similar ease. In this work, we propose a novel approach to disentangle the cosine similarity of mean embeddings into the product of cosine similarities for individual pixel clusters via central kernel alignment. Consequently, we can quantify the contribution of the cluster-wise performance to the overall image generation performance. We demonstrate how this enhances the explainability and the likelihood of identifying pixel regions of model misbehavior across various real-world use cases.

View on arXivView PDF

Code Implementations(1)

MLO-lab/Disentangling_Mean_EmbeddingsCommunity100%

Codebase for "Disentangling Mean Embeddings for Better Diagnostics of Image Generators", published at IAI Workshop @ NeurIPS 2024.

10Python, Jupyter NotebookNov 22, 20241 years ago
deep-learningexplainable-aiimage-generationkernelsmachine-learning

Similar Papers

Towards an Explainable Comparison and Alignment of Feature Embeddings

78% match1 repo182025

From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature Selection

77% match1 repo42025

CSIM: A Copula-based similarity index sensitive to local changes for Image quality assessment

76% match1 repo112024

Layer-Wise Feature Metric of Semantic-Pixel Matching for Few-Shot Learning

76% match1 repo32024

Scendi Score: Prompt-Aware Diversity Evaluation via Schur Complement of CLIP Embeddings

76% match1 repo102024

Image Similarity using An Ensemble of Context-Sensitive Models

76% match1 repo22024

Cite this paper

@article{gruber2024disentangling,
  title  = {Disentangling Mean Embeddings for Better Diagnostics of Image Generators},
  author = {Sebastian G. Gruber and Pascal Tobias Ziegler and Florian Buettner},
  year   = {2024},
  eprint = {2409.01314},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.01314v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.01314.svg)](https://codeofpaper.com/paper/2409.01314)

Discussion