Automated Monitoring of Cultural Heritage Artifacts Using Semantic Segmentation

2025-11-25

AI Summary

Automated detection and segmentation of cracks in cultural heritage artifacts (e.g., statues, monuments) remain challenging due to poor generalizability across diverse, unseen artifact types and conditions. To address this, we propose a semantic segmentation framework based on the U-Net architecture, systematically evaluating multiple CNN encoders—including ResNet and EfficientNet—for fine-grained, pixel-level crack delineation. Evaluated on the OmniCrack30k dataset, our models achieve state-of-the-art performance in mIoU, Dice, and Jaccard scores. Crucially, we demonstrate that pre-trained encoders exhibit strong cross-domain generalization to previously unseen heritage objects without artifact-specific fine-tuning. Furthermore, out-of-distribution qualitative validation on unlabeled real-world damaged images confirms robustness under varying lighting conditions, material textures, and scale variations. This work establishes a plug-and-play, high-accuracy, and highly adaptable paradigm for automated crack diagnosis in cultural heritage conservation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Deep Generative Models & Autoencoders
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

This paper addresses the critical need for automated crack detection in the preservation of cultural heritage through semantic segmentation. We present a comparative study of U-Net architectures, using various convolutional neural network (CNN) encoders, for pixel-level crack identification on statues and monuments. A comparative quantitative evaluation is performed on the test set of the OmniCrack30k dataset [1] using popular segmentation metrics including Mean Intersection over Union (mIoU), Dice coefficient, and Jaccard index. This is complemented by an out-of-distribution qualitative evaluation on an unlabeled test set of real-world cracked statues and monuments. Our findings provide valuable insights into the capabilities of different CNN- based encoders for fine-grained crack segmentation. We show that the models exhibit promising generalization capabilities to unseen cultural heritage contexts, despite never having been explicitly trained on images of statues or monuments.

Problem

• Automated crack detection for cultural heritage preservation using semantic segmentation
• Comparative evaluation of U-Net architectures with CNN encoders for crack identification
• Assessing model generalization capabilities on unseen cultural heritage contexts

Innovation

• U-Net architectures with CNN encoders
• Pixel-level crack identification on monuments
• Comparative evaluation using segmentation metrics