Archival Faces: Detection of Faces in Digitized Historical Documents

2025-04-01

AI Summary

Historical scanned documents—such as 19th–20th century newspapers—pose severe challenges for face detection, with existing detectors achieving only ~24% mAP (IoU 0.5–0.9), insufficient for face–text cross-modal retrieval. To address this, we introduce HFaces, the first high-quality, domain-specific face dataset for historical documents, comprising 2.2k images and 11k bounding boxes with facial landmark annotations. Following the WIDERFace annotation protocol, all instances were manually refined. We systematically adapt and fine-tune three state-of-the-art detectors—Faster R-CNN (with multi-scale training), YOLOv8, and RetinaFace—and conduct ablation studies to identify optimal configurations. Experimental results demonstrate substantial improvements in detection accuracy (mAP significantly surpassing baseline methods) and robust facial landmark prediction. HFaces establishes a new benchmark for historical document analysis, and our optimized models provide a reliable foundation for face–text joint retrieval in archival applications.

Technology Category

• Computer Vision: Image and Video Retrieval
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

When digitizing historical archives, it is necessary to search for the faces of celebrities and ordinary people, especially in newspapers, link them to the surrounding text, and make them searchable. Existing face detectors on datasets of scanned historical documents fail remarkably -- current detection tools only achieve around $24%$ mAP at $50:90%$ IoU. This work compensates for this failure by introducing a new manually annotated domain-specific dataset in the style of the popular Wider Face dataset, containing 2.2k new images from digitized historical newspapers from the $19^{th}$ to $20^{th}$ century, with 11k new bounding-box annotations and associated facial landmarks. This dataset allows existing detectors to be retrained to bring their results closer to the standard in the field of face detection in the wild. We report several experimental results comparing different families of fine-tuned detectors against publicly available pre-trained face detectors and ablation studies of multiple detector sizes with comprehensive detection and landmark prediction performance results.

Problem

• Improving face detection in historical documents
• Creating annotated dataset for retraining detectors
• Enhancing searchability of faces in archives

Innovation

• Introduces manually annotated historical face dataset
• Retrains detectors to improve historical face recognition
• Compares fine-tuned and pre-trained detector performances