RA-ClipScore: Making Generative Model Evaluation More Interpretable

2026-08-12

AI Summary

This work addresses the limited interpretability of existing evaluation metrics for generative models, which struggle to diagnose generation biases at both semantic attribute and spatial distribution levels. To this end, the paper proposes RA-CLIPScore, the first CLIP-based evaluation framework extended to incorporate spatial alignment. RA-CLIPScore employs a dual-prompt mechanism to disentangle competing attributes and leverages local image patch tokens to model region-wise semantic alignment. By introducing a region-specific single-attribute divergence metric, the method substantially enhances interpretability and aligns more closely with human perception. Experiments demonstrate that RA-CLIPScore exhibits greater robustness under distribution shifts or when textual prompts contain partially irrelevant attributes, and its scores show strong correlation with human judgments of visual diversity.

Abstract

Generative models can produce images nearly indistinguishable from real data, yet rigorous and interpretable evaluation remains challenging. Conventional metrics such as FID provide only scalar scores with limited diagnostic insight. Widely adopted CLIP-based metrics enable semantic evaluation beyond simple training class labels, but inherit limitations from CLIP's training paradigm that restrict attribute-wise analysis. We propose RA-CLIPScore, a novel metric that mitigates these issues and extends CLIP-based evaluation to spatial distribution alignment, measuring whether generated objects adhere to the positional priors found in the training data. RA-CLIPScore introduces dual prompts to decouple competing attributes and leverages local patch tokens to capture fine-grained regional semantics. We evaluate image generative models on their ability to match both attribute and spatial distributions of the training data. Extensive experiments show that RA-CLIPScore provides more robust and interpretable evaluations than prior methods, particularly under distribution misalignment or partially irrelevant textual attributes. We further demonstrate how it reveals spatial biases in generative models. User evaluations confirm that Regional Single Attribute Divergence based on our RA-CLIPScore aligns more closely with human perception of visual diversity than existing semantic metrics.

Problem

• generative model evaluation
• interpretability
• spatial distribution alignment
• attribute-wise analysis
• CLIP-based metrics

Innovation

• RA-CLIPScore
• spatial distribution alignment
• dual prompts
• local patch tokens
• interpretable evaluation