Augmented Reality in Cultural Heritage: A Dual-Model Pipeline for 3D Artwork Reconstruction

2025-07-18

AI Summary

Addressing the dual challenges of single-image artwork recognition and high-fidelity 3D reconstruction in museum settings, this paper proposes a dual-model collaborative depth estimation framework that synergistically integrates GLPN (for global structural modeling) and Depth-Anything (for local detail enhancement) to achieve end-to-end depth map optimization. Subsequently, geometrically accurate and visually realistic 3D models are reconstructed via point cloud generation, Poisson surface reconstruction, and texture mapping, and seamlessly integrated into a lightweight AR system. Compared to single-model baselines, the proposed method reduces reconstruction error by 23.6% on complex artistic features—such as sculptures and reliefs—and significantly improves texture consistency. Experimental results demonstrate real-time, stable, and highly immersive in-museum AR interaction. The approach provides a scalable, low-barrier technical pathway for digital heritage visualization and dissemination.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

This paper presents an innovative augmented reality pipeline tailored for museum environments, aimed at recognizing artworks and generating accurate 3D models from single images. By integrating two complementary pre-trained depth estimation models, i.e., GLPN for capturing global scene structure and Depth-Anything for detailed local reconstruction, the proposed approach produces optimized depth maps that effectively represent complex artistic features. These maps are then converted into high-quality point clouds and meshes, enabling the creation of immersive AR experiences. The methodology leverages state-of-the-art neural network architectures and advanced computer vision techniques to overcome challenges posed by irregular contours and variable textures in artworks. Experimental results demonstrate significant improvements in reconstruction accuracy and visual realism, making the system a highly robust tool for museums seeking to enhance visitor engagement through interactive digital content.

Problem

• Reconstruct 3D artwork models from single museum images
• Combine depth models for global and local feature accuracy
• Enhance AR experiences with high-quality point clouds and meshes

Innovation

• Integrates GLPN and Depth-Anything models
• Converts depth maps to point clouds
• Uses neural networks for artwork reconstruction