Problem
Research questions and friction points this paper is trying to address.
digitization
historical documents
OCR errors
retrieval-augmented generation
factual accuracy
A Comparative Evaluation of Digitization Pipelines for Historiographical Sources
🤖 AI Summary
研究评估了历史文献数字化流程，采用直接提取、大语言模型后校正及分块提取法，解决了OCR错误传播问题，提高信息检索准确性。
📝 Abstract
Purpose: The digitization of historical documents presents fundamental challenges for modern information retrieval and Artificial Intelligence (AI) systems. Optical character recognition (OCR) errors in source corpora propagate through retrieval-augmented generation (RAG) pipelines, compromising the factual accuracy of generated outputs. Methods: This study presents a systematic evaluation of PDF-to-text extraction pipelines applied to historiographical secondary sources on the Visigothic period. We assess thirteen distinct approaches spanning three methodological families: direct extraction, Large Language Model (LLM) post-correction, and chunk-and-extract. Documents are stratified into five categories based on production method and visual complexity. Performance is measured using character error rate (CER) and word error rate (WER) against manually corrected ground truth. Results: Results demonstrate that direct extraction with Marker achieves superior performance (98.70% CER accuracy; 97.71% WER accuracy overall), while conventional OCR pipelines exhibit substantial degradation on scanned documents and complex layouts. Embedded-text extraction performs well on digital PDFs but fails on scanned documents. LLM post-correction does not provide systematic improvements and frequently degrades accurate extractions. Conclusion: End-to-end document parsing is the most reliable approach for heterogeneous historical collections. Document characteristics such as scan quality, layout complexity, and the presence of embedded text layers have a significant impact on extraction accuracy. LLM-based post-correction should not be assumed beneficial by default and requires validation before large-scale application.
Innovation
Methods, ideas, or system contributions that make the work stand out.
direct extraction
historical documents digitization
LLM post-correction
end-to-end document parsing
🔎 Similar Papers
2024-01-30Citations: 0
