When Simpler Is Better: Evaluating Translation Pipelines for Medieval Latin Manuscripts

2026-07-04

AI Summary

This study addresses the challenges of machine translation for medieval Latin manuscripts, including low-resource transcription, archaic language recognition, and image noise. The authors introduce the first end-to-end image-to-translation evaluation framework and release the Interpres-Parallel-Corpus, a dataset comprising 1,383 aligned lines of manuscript images, transcriptions, and expert translations. Through systematic comparison of domain-specific OCR, vision-language models (VLMs), retrieval-augmented generation (RAG), and post-OCR correction strategies, they find that a minimal OCR+VLM pipeline yields the highest translation quality, whereas more complex multi-component architectures suffer from error propagation and prompt saturation. Experiments reveal that domain-specific OCR reduces character error rates by up to 4.3× compared to general-purpose VLMs, highlighting a “complexity paradox” wherein simplified pipelines prove more effective.

Abstract

Despite remarkable progress in machine translation, Vision Language Models (VLMs) struggle on historical manuscripts, a domain that stresses core Natural Language Processing (NLP) capabilities: low-resource transliteration, archaic vocabulary, and noisy input signals. We present a systematic framework for evaluating the full image-to-translation pipeline on medieval Latin manuscripts, a setting in which scribal shorthand, ligatures, and parchment degradation expose failure modes that are invisible in clean-text benchmarks. Benchmarking on the CATMuS Latin dataset reveals a specialization gap: domain-specific Optical Character Recognition (OCR) models reduce character error rate by up to 4.3$\times$ compared to general-purpose VLMs, despite operating at orders of magnitude fewer parameters. We introduce the Interpres-Parallel-Corpus (IPC), a novel dataset comprising 1,383 aligned manuscript image lines, transcriptions, and expert translations, the first of its kind for medieval Latin. Our experiments uncover a complexity paradox: the simplest pipeline, a specialized OCR model feeding directly into a VLM, outperforms all multi-component variants. Adding retrieval-augmented generation (RAG) or post-OCR correction introduces prompt saturation and error propagation that degrade aggregate translation quality. These findings offer both a new benchmark and practical guidance for deploying translation systems in low-resource historical settings.

Problem

• medieval Latin manuscripts
• machine translation
• low-resource
• OCR
• historical documents

Innovation

• Vision Language Models
• Optical Character Recognition
• Medieval Latin
• Translation Pipeline
• Low-resource NLP