RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wild

2026-06-22

AI Summary

This work addresses the challenges of low accuracy and inefficiency in layout analysis caused by the heterogeneity of document elements, geometric deformations, and the entanglement of reading order in complex documents. To this end, we propose an end-to-end, single-stage multi-task framework based on RT-DETR that jointly models geometric and structural information within a unified query decoder. The model simultaneously performs element classification, bounding box detection, pixel-level segmentation, and reading order prediction. Evaluated on public benchmarks, our approach achieves state-of-the-art performance with an inference speed of 132.1 FPS and significantly enhances the quality of full-text reconstruction in downstream OCR tasks.

Abstract

Accurate document layout analysis remains a critical bottleneck for document parsing systems, due to the intricate coupling among heterogeneous document layout elements, geometric distortions (\eg, paper warping and bending, perspective variations), and reading order within diverse layout structures. Existing approaches typically rely on fragmented multi-stage pipelines or computationally heavy generative Transformer architectures, leading to error propagation and limited efficiency. In this paper, we present RT-DocLayout, a highly efficient end-to-end framework for document layout analysis, designed as a front-end for document parsing tasks. The proposed model unifies classification, detection, pixel-level segmentation, and reading order prediction for layout elements within a single 33M-parameter architecture. Built upon the RT-DETR, our key contribution is a unified multi-task formulation within a single query-based decoder that simultaneously classifies, regresses bounding box, generates masks, and constructs relationship to reason reading order. By jointly learning geometric and structural representations, RT-DocLayout introduces multi-task optimization that substantially improves robustness under real-world document distortions. Extensive experiments on public benchmarks demonstrate state-of-the-art performance in document layout analysis while maintaining real-time inference speed(132.1 FPS). When coupled with downstream OCR engines, RT-DocLayout significantly improves full-document reconstruction quality, providing a scalable and practical foundation for real-world document intelligence systems.

Problem

• document layout analysis
• reading order
• geometric distortions
• heterogeneous layout elements
• real-time inference

Innovation

• end-to-end document layout analysis
• reading order prediction
• multi-task learning
• real-time inference
• query-based decoder