Cross-Cultural Expert-Level Art Critique Evaluation with Vision-Language Models

Abstract

Vision-Language Models (VLMs) excel at visual perception, yet their ability to interpret cultural meaning in art remains under-validated. However, cultural understanding and interpretability are often overlooked when evaluating these models. To overcome this limitation, this paper introduces a tri-tier evaluation framework for cross-cultural art-critique assessment. Tier I provides a series of automated metrics indicating cultural coverage. Tier II leverages theory-informed template-based scoring using a single primary judge across five evaluation dimensions (Coverage, Alignment, Depth, Accuracy, Quality), each rated on a 1--5 scale. Tier III then calibrates the aggregated scores from Tier II via isotonic regression. The proposed evaluation framework is validated with a large-scale experiment covering 15 different VLMs on 294 evaluation art-critique pairs spanning six different cultural traditions. Our findings reveal that (i) automated metrics are unreliable for cultural depth analysis, (ii) Western samples score higher than non-Western samples under our sampling and evaluation template, highlighting potential model biases, and (iii) VLMs exhibit a consistent performance gap, performing well in visual description but underperforming in cultural interpretation. Dataset and code are available at https://github.com/yha9806/VULCA-Framework.