MoDora: Tree-Based Semi-Structured Document Analysis System

2026-02-26

AI Summary

This work addresses the limitations of existing methods in question answering over semi-structured documents containing tables, figures, and hierarchical paragraphs, which often suffer from fragmented OCR outputs, inadequate hierarchical modeling, and insufficient cross-region alignment. To overcome these challenges, we propose MoDora, a novel system that reconstructs OCR results through local alignment aggregation, explicitly models inter-component hierarchical and spatial relationships via a Component Correlation Tree (CCTree), and introduces a question-type-aware hybrid retrieval mechanism that fuses semantic and layout information. Evaluated on multiple benchmarks, MoDora significantly outperforms current state-of-the-art approaches, achieving absolute accuracy gains of 5.97% to 61.07% and enabling precise, layout- and semantics-driven document understanding.

Technology Category

• Natural Language Processing: Question Answering
• Computer Vision: Multi-modal Vision
• Machine Learning: Mixture of Experts (MoE)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Semi-structured documents integrate diverse interleaved data elements (e.g., tables, charts, hierarchical paragraphs) arranged in various and often irregular layouts. These documents are widely observed across domains and account for a large portion of real-world data. However, existing methods struggle to support natural language question answering over these documents due to three main technical challenges: (1) The elements extracted by techniques like OCR are often fragmented and stripped of their original semantic context, making them inadequate for analysis. (2) Existing approaches lack effective representations to capture hierarchical structures within documents (e.g., associating tables with nested chapter titles) and to preserve layout-specific distinctions (e.g., differentiating sidebars from main content). (3) Answering questions often requires retrieving and aligning relevant information scattered across multiple regions or pages, such as linking a descriptive paragraph to table cells located elsewhere in the document. To address these issues, we propose MoDora, an LLM-powered system for semi-structured document analysis. First, we adopt a local-alignment aggregation strategy to convert OCR-parsed elements into layout-aware components, and conduct type-specific information extraction for components with hierarchical titles or non-text elements. Second, we design the Component-Correlation Tree (CCTree) to hierarchically organize components, explicitly modeling inter-component relations and layout distinctions through a bottom-up cascade summarization process. Finally, we propose a question-type-aware retrieval strategy that supports (1) layout-based grid partitioning for location-based retrieval and (2) LLM-guided pruning for semantic-based retrieval. Experiments show MoDora outperforms baselines by 5.97%-61.07% in accuracy. The code is at https://github.com/weAIDB/MoDora.

Problem

• semi-structured documents
• natural language question answering
• hierarchical structure
• layout-aware representation
• information retrieval

Innovation

• Component-Correlation Tree
• layout-aware document analysis
• semi-structured document QA
• LLM-guided retrieval
• hierarchical document representation