A Semi-Automated Framework for 3D Reconstruction of Medieval Manuscript Miniatures

2026-04-08

AI Summary

This work addresses the challenge of efficiently converting medieval manuscript miniatures into high-quality 3D models by proposing a hybrid reconstruction paradigm that integrates artificial intelligence with expert intervention. The approach combines SAM-based image segmentation, Hi3DGen for initial mesh generation, manual refinement in ZBrush, and AI-assisted texture mapping. It presents the first systematic evaluation of multiple image-to-3D methods on historical manuscript imagery, validated through multidimensional metrics including Silhouette IoU, LPIPS, CLIP Score, and volumetric error. The pipeline successfully reconstructed 69 manuscript images, yielding models compatible with WebXR, AR overlay, and haptic 3D printing. Demonstrated on the *Decretum Gratiani* and works by Giulio Clovio, the method exhibits cross-traditional applicability while preserving both geometric fidelity and topological integrity.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Humans and AI: Crowd Sourcing and Human Computation

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

This paper presents a semi-automated framework for transforming two-dimensional miniatures from medieval manuscripts into three-dimensional digital models suitable for extended reality (XR), tactile 3D~printing, and web-based visualization. We evaluate seven image-to-3D methods (TripoSR, SF3D, SPAR3D, TRELLIS, Wonder3D, SAM~3D, Hi3DGen) on 69~manuscript figures from two collections using rendering-based metrics (Silhouette IoU, LPIPS, CLIP~Score) and volumetric measures (Depth Range Ratio, watertight percentage), revealing a trade-off between volumetric expansion and geometric fidelity. Hi3DGen balances topological quality with rich surface detail through its normal bridging approach, making it a good starting point for expert refinement. Our pipeline combines SAM segmentation, Hi3DGen mesh generation, expert refinement in ZBrush, and AI-assisted texturing. Two case studies on Gothic illuminations from the Decretum Gratiani (Vatican Library) and Renaissance miniatures by Giulio Clovio demonstrate applicability across artistic traditions. The resulting models can support WebXR visualization, AR overlay on physical manuscripts, and tactile 3D~prints for visually impaired users.

Problem

• 3D reconstruction
• medieval manuscript miniatures
• extended reality
• tactile 3D printing
• digital heritage

Innovation

• 3D reconstruction
• medieval manuscripts
• semi-automated pipeline
• Hi3DGen
• extended reality