Scaling State-Space Models from Lines to Paragraphs: An Ablation of Mamba-based OCR

2026-06-22

AI Summary

This study addresses the challenge of efficiently scaling optical character recognition (OCR) from single-line to paragraph-level text, where traditional Transformers suffer from quadratic complexity due to their attention mechanism, and the applicability of state space models like Mamba to long sequences and handwritten text remains unclear. The work presents the first systematic investigation of Mamba as an autoregressive decoder in OCR, employing controlled ablation studies to analyze the impact of key hyperparameters such as state dimension and expansion factor. Results show that on synthetic paragraph data, Mamba achieves a character error rate (CER) below 1% and inference speeds 1.4–4.5× faster than Transformer. However, on real-world handwritten data (IAM), its performance lags significantly (CER: 10.0% vs. 3.5%), indicating that the primary bottleneck stems from data scarcity rather than architectural limitations.

Abstract

End-to-end OCR increasingly relies on autoregressive sequence models, where the quadratic cost of Transformer attention limits efficient transcription of long, paragraph-level text. State-Space Models (SSMs) such as Mamba offer linear-time decoding and have recently been shown to match Transformer accuracy on printed historical lines, but their behavior as sequences grow from short lines to full paragraphs, and their generalization to handwriting, remain poorly understood. We study how a Mamba-based OCR recognizer scales from lines to paragraphs. We first conduct a systematic exploration of its four core hyperparameters (decoder depth, state dimension, expansion factor, and connector depth) on synthetic paragraphs from 100 to 1,000 characters, identifying the recurrent state dimension and the expansion factor as the dominant levers for long-sequence accuracy. We then compare the recognizer against a Transformer baseline trained under an identical protocol. On clean synthetic paragraphs, both models stay below 1% CER at every length while the SSM runs 1.4 to 4.5 times faster, the speedup growing with sequence length. On real handwriting, however, the SSM lags clearly behind: it reaches 8.2% CER on IAM lines and 10.0% on IAM paragraphs, against 4.2% and 3.5% for the Transformer baseline. Through controlled experiments we show that a substantial part of this gap stems from data scarcity rather than from an intrinsic architectural limit: the autoregressive SSM decoder is markedly data-hungry on long sequences. Our study clarifies when SSMs are a practical choice for large-scale document transcription and when they are not.

Problem

• State-Space Models
• OCR
• long-sequence transcription
• handwriting recognition
• data scarcity

Innovation

• State-Space Models
• Mamba
• OCR
• long-sequence modeling
• data efficiency