Artificial Intelligence and Misinformation in Art: Can Vision Language Models Judge the Hand or the Machine Behind the Canvas?

2025-08-02

AI Summary

This study addresses the unreliability of vision-language models (VLMs) in two critical art-domain tasks: painter attribution and AI-generated image detection. We conduct the first systematic evaluation of state-of-the-art VLMs on both tasks, using a high-quality dataset comprising nearly 40,000 artworks by 128 artists, alongside contemporary image generation and analysis models. Results reveal severe performance limitations: VLMs exhibit low accuracy and systematic biases—including misclassifying human-made paintings as AI-generated and mistaking early stylized AI outputs for human creations—thereby exacerbating misinformation risks in art contexts. Our core contributions are threefold: (1) uncovering structural deficiencies in VLMs’ artistic semantic understanding; (2) proposing a novel evaluation paradigm tailored to art-specific trustworthiness assessment; and (3) providing empirical evidence and actionable insights to mitigate authenticity crises induced by generative AI in visual arts.

Technology Category

• Computer Vision: Large Vision Models
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

The attribution of artworks in general and of paintings in particular has always been an issue in art. The advent of powerful artificial intelligence models that can generate and analyze images creates new challenges for painting attribution. On the one hand, AI models can create images that mimic the style of a painter, which can be incorrectly attributed, for example, by other AI models. On the other hand, AI models may not be able to correctly identify the artist for real paintings, inducing users to incorrectly attribute paintings. In this paper, both problems are experimentally studied using state-of-the-art AI models for image generation and analysis on a large dataset with close to 40,000 paintings from 128 artists. The results show that vision language models have limited capabilities to: 1) perform canvas attribution and 2) to identify AI generated images. As users increasingly rely on queries to AI models to get information, these results show the need to improve the capabilities of VLMs to reliably perform artist attribution and detection of AI generated images to prevent the spread of incorrect information.

Problem

• Can AI models accurately attribute paintings to correct artists?
• Can AI models detect AI-generated art mimicking human styles?
• Do vision language models reliably prevent misinformation in art attribution?

Innovation

• Using vision language models for art attribution
• Testing AI on 40,000 paintings from 128 artists
• Evaluating AI's ability to detect generated images