🤖 AI Summary
This work addresses the multifaceted challenges of ancient Greek text processing—including textual restoration, word segmentation, accentuation, punctuation recovery, and morphosyntactic analysis—by proposing a character-level masked diffusion encoder with 405 million parameters. The model innovatively decomposes input into five independently maskable, aligned planes (letters, word/sentence boundaries, diacritics, case, and punctuation), enabling a unified architecture that handles all tasks without task-specific tokenization. Pretrained on an open corpus of 380 million words and rigorously evaluated using a ten-fold decontaminated cross-validation protocol to ensure generalizability, the model significantly outperforms state-of-the-art methods: it reduces character error rate in epigraphic restoration to 15.5% (a 9.1 percentage-point improvement), increases labeled attachment score (LAS) in dependency parsing by 12.9, boosts balanced accuracy in macron annotation by 6.0 points, and achieves 74.5% top-1 accuracy on the Ithaca benchmark.
📝 Abstract
We introduce Stoicheia, a 405M-parameter character-level masked-diffusion encoder for Ancient Greek whose input factors into five aligned, independently maskable planes: letters, word and sentence boundaries, diacritics, capitalization, and punctuation. A single backbone can therefore restore lacunae, re-segment, accentuate, and punctuate unspaced text without task-specific retokenization. We pretrain it on an open, revision-pinned corpus of 380M words and release eleven checkpoints: ten rotated, decontaminated folds, guaranteeing that for any given literary passage at least one released model has never seen its text, and one with no exposure to documentary texts. Three experiments - reconstruction of damaged inscriptions and papyri, morphosyntactic tagging and dependency parsing, and macronization with metrical scansion - each carry a matched random-initialization control, isolating what character-level diffusion pretraining contributes: 5.6 CER points on inscription reconstruction, 12.9 LAS on parsing, and 6.0 points of balanced accuracy on macronization. On Ithaca's own test split, with identical frozen samples and strict scoring, Stoicheia reduces character error relative to both prior state-of-the-art systems, from 24.6 (Ithaca) and 23.5 (its 2025 Aeneas-framework successor) to 15.5, and raises top-1 accuracy from 63.0 and 64.0 to 74.5.