QUEST: Quality-aware Semi-supervised Table Extraction for Business Documents

2025-06-17

AI Summary

To address the challenges of sparse human annotations, multi-stage error accumulation, and unreliable pseudo-labels in commercial document table extraction (TE), this paper proposes a quality-aware semi-supervised framework. Methodologically, it introduces: (1) a novel F1-predictability assessment model grounded in structural and contextual modeling for fine-grained quality quantification; (2) a row- and column-level diversity sampling mechanism integrating Determinantal Point Processes (DPP), Vendi Score, and IntDiv to mitigate confirmation bias; and (3) an interpretable, robust quality-guided pseudo-label filtering strategy. Evaluated on a proprietary business dataset, the framework achieves an F1 score of 74% (+10 percentage points) and reduces empty-table false detection by 45%. On the DocILE benchmark, it attains an F1 score of 50% (+8 percentage points) and lowers empty-table prediction errors by 19%.

Technology Category

• Natural Language Processing: Information Extraction
• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

Automating table extraction (TE) from business documents is critical for industrial workflows but remains challenging due to sparse annotations and error-prone multi-stage pipelines. While semi-supervised learning (SSL) can leverage unlabeled data, existing methods rely on confidence scores that poorly reflect extraction quality. We propose QUEST, a Quality-aware Semi-supervised Table extraction framework designed for business documents. QUEST introduces a novel quality assessment model that evaluates structural and contextual features of extracted tables, trained to predict F1 scores instead of relying on confidence metrics. This quality-aware approach guides pseudo-label selection during iterative SSL training, while diversity measures (DPP, Vendi score, IntDiv) mitigate confirmation bias. Experiments on a proprietary business dataset (1000 annotated + 10000 unannotated documents) show QUEST improves F1 from 64% to 74% and reduces empty predictions by 45% (from 12% to 6.5%). On the DocILE benchmark (600 annotated + 20000 unannotated documents), QUEST achieves a 50% F1 score (up from 42%) and reduces empty predictions by 19% (from 27% to 22%). The framework's interpretable quality assessments and robustness to annotation scarcity make it particularly suited for business documents, where structural consistency and data completeness are paramount.

Problem

• Automating table extraction from business documents with sparse annotations
• Improving semi-supervised learning by assessing table extraction quality
• Reducing empty predictions and enhancing structural consistency in extracted tables

Innovation

• Quality-aware SSL framework for table extraction
• Novel F1-predicting quality assessment model
• Diversity measures mitigate confirmation bias