Does AI See like Art Historians? Interpreting How Vision Language Models Recognize Artistic Style

2026-03-11

AI Summary

This study investigates whether the mechanisms underlying visual language models’ (VLMs) recognition of artistic styles align with the criteria employed by art historians. By decomposing the model’s latent space to isolate visual concepts driving style predictions and integrating causal intervention analysis with expert evaluations from art historians, the work pioneers a deep integration of art historical knowledge into AI interpretability research. The findings reveal that 73% of the extracted concepts are judged by experts as semantically clear and visually coherent, while 90% of the concepts used for style prediction are deemed relevant. Moreover, certain seemingly irrelevant yet effective concepts receive plausible interpretations, uncovering both alignments and discrepancies between how VLMs and human experts understand artistic style.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

VLMs have become increasingly proficient at a range of computer vision tasks, such as visual question answering and object detection. This includes increasingly strong capabilities in the domain of art, from analyzing artwork to generation of art. In an interdisciplinary collaboration between computer scientists and art historians, we characterize the mechanisms underlying VLMs' ability to predict artistic style and assess the extent to which they align with the criteria art historians use to reason about artistic style. We employ a latent-space decomposition approach to identify concepts that drive art style prediction and conduct quantitative evaluations, causal analysis and assessment by art historians. Our findings indicate that 73% of the extracted concepts are judged by art historians to exhibit a coherent and semantically meaningful visual feature and 90% of concepts used to predict style of a given artwork were judged relevant. In cases where an irrelevant concept was used to successfully predict style, art historians identified possible reasons for its success; for example, the model might "understand" a concept in more formal terms, such as dark/light contrasts.

Problem

• vision language models
• artistic style
• art historians
• style recognition
• visual semantics

Innovation

• vision-language models
• artistic style recognition
• latent-space decomposition
• interdisciplinary evaluation
• concept interpretability