Integrating Visual and X-Ray Machine Learning Features in the Study of Paintings by Goya

2025-11-02

AI Summary

Authenticating Goya’s artworks is challenging due to their highly heterogeneous stylistic features and frequent historical forgeries. Method: This paper proposes a multimodal machine learning framework that jointly analyzes visible-light and X-ray imaging data. It introduces a unified feature extraction pipeline integrating gray-level co-occurrence matrices (GLCM), local binary patterns (LBP), entropy/energy features, color distribution statistics, and one-class support vector machines (OCSVM) for joint modeling and anomaly detection. Contribution/Results: By synergistically leveraging complementary information across modalities—particularly latent material-layer evidence inaccessible to single-modality analysis—the framework achieves 97.8% classification accuracy and a 2.2% false positive rate on a dataset of 24 authenticated Goya masterpieces. It further attains a 92.3% confidence score in authenticating the contested work *Un Gigante*. The approach significantly enhances the robustness and interpretability of non-invasive, scientific attribution for high-value artworks.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Security and Privacy: Data transparency and provenance
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Art authentication of Francisco Goya's works presents complex computational challenges due to his heterogeneous stylistic evolution and extensive historical patterns of forgery. We introduce a novel multimodal machine learning framework that applies identical feature extraction techniques to both visual and X-ray radiographic images of Goya paintings. The unified feature extraction pipeline incorporates Grey-Level Co-occurrence Matrix descriptors, Local Binary Patterns, entropy measures, energy calculations, and colour distribution analysis applied consistently across both imaging modalities. The extracted features from both visual and X-ray images are processed through an optimised One-Class Support Vector Machine with hyperparameter tuning. Using a dataset of 24 authenticated Goya paintings with corresponding X-ray images, split into an 80/20 train-test configuration with 10-fold cross-validation, the framework achieves 97.8% classification accuracy with a 0.022 false positive rate. Case study analysis of ``Un Gigante'' demonstrates the practical efficacy of our pipeline, achieving 92.3% authentication confidence through unified multimodal feature analysis. Our results indicate substantial performance improvement over single-modal approaches, establishing the effectiveness of applying identical computational methods to both visual and radiographic imagery in art authentication applications.

Problem

• Authenticating Goya paintings with multimodal machine learning
• Applying identical feature extraction to visual and X-ray images
• Improving authentication accuracy over single-modal approaches

Innovation

• Multimodal machine learning framework for visual and X-ray images
• Identical feature extraction pipeline applied across imaging modalities
• Optimized One-Class SVM processing unified multimodal features