AlignFace: Human-Aligned Face Similarity Metric with Interpretable Concept Relations

2026-08-14

AI Summary

This study addresses the lack of cognitive alignment and neglect of population heterogeneity in existing face similarity metrics by proposing an interpretable, human-perception-aligned method grounded in cognitive psychology. The approach explicitly models cognitive mechanisms—including facial features, nonlinear responses, and group biases—and integrates vision-language models, gated cross-attention, concept bottlenecks, and neural generalized additive models for joint optimization. Experimental results on the FACETS dataset demonstrate that the proposed method significantly outperforms state-of-the-art metrics, effectively enhancing both alignment accuracy with human perceptual judgments across diverse populations and model interpretability. This work bridges computational modeling and cognitive science to advance more psychologically plausible face representation learning.

Abstract

Computer vision models for generated facial content, such as face editing and privacy protection, increasingly affect people, requiring similarity metrics that serve as faithful proxies for human perception. While perceptual evaluation has progressed from signal-based heuristics to representation-based metrics, current approaches are limited to behavioral modeling without cognitive alignment. They rely on implicit and spurious relations while assuming a universal observer, failing to account for inherent variations across diverse human populations. This leads to inaccurate evaluative models of stakeholders and misleading guidance for generative model debugging. Rather than treating perception as a black box, we leverage scientific findings from cognitive psychology of human face similarity perception: dependence on facial featural and configural attributes, nonlinear psychophysical response scaling, and own-group biases. We introduce the FACETS dataset and propose AlignFace, an interpretable, human-aligned, face similarity metric that encodes these cognitive principles through ante-hoc modeling. It employs visual-language modeling (VLM) to encode paired face images and text-based attributes, gated cross-attention (CA) to extract attribute-specific facial difference representations, concept bottleneck modeling (CBM) to constrain reasoning via interpretable face attributes, and neural generalized additive model (GAM) to model their nonlinear influence. Experiments found AlignFace significantly improves alignment with human subpopulation perceptions compared to baseline metrics, including recent domain-free learned perceptual metrics. By bridging learned representations and human cognitive processes, this work enables more transparent and aligned perceptual evaluation metrics for face images.

Problem

• Face Similarity Metric
• Human-Aligned Perception
• Cognitive Alignment
• Interpretability
• Population Variation

Innovation

• Human-Aligned Metric
• Concept Bottleneck Modeling
• Neural Generalized Additive Model
• Cognitive Psychology Integration
• Interpretable Face Similarity