Active Learning for Cascaded Object Detection: Balancing Coverage and Uncertainty in Table Extraction Pipelines

2026-07-01

AI Summary

This work addresses the high annotation cost in the structure recognition phase of cascaded table extraction and the neglect of inter-stage dependencies between detection and recognition in existing active learning approaches. To this end, it introduces Uncertainty Herding into this pipeline for the first time and proposes two pipeline-aware active learning methods, RankFusion and CAPA. These methods jointly model representativeness and uncertainty by integrating dual-space coverage from both detection and structural representation, incorporating a stage-aware gating mechanism, and calibrating task-specific uncertainty, thereby explicitly leveraging cross-stage dependencies. Experimental results demonstrate that CAPA significantly outperforms baseline methods on three out of four benchmark datasets, effectively reducing annotation costs while enhancing model performance.

Abstract

Table extraction from business documents relies on a cascaded pipeline where Table Detection (TD) first localizes tables and Table Structure Recognition (TSR) then recovers their internal layout. Building task-specific training sets for this pipeline is costly, particularly for TSR which requires fine-grained structural annotations. Active learning (AL) can reduce this annotation burden, yet most AL strategies are designed for single-model tasks and do not account for inter-stage dependencies in cascaded architectures. In this work, we present the first adaptation of Uncertainty Herding (UHerding), a hybrid coverage-uncertainty sampling method originally proposed for image classification, to cascaded object detection pipelines. We propose two pipeline-aware extensions that exploit the TD-to-TSR dependency: RankFusion adds dual-manifold coverage over both detection and structure representation spaces, while CAPA further incorporates stage-dependent gating and per-task uncertainty calibration. Extensive experiments across two public (PubTables-1M and FinTabNet) and two private table extraction datasets, with various annotation budgets (from 71 to 500 documents) show that UHerding generalizes well to table extraction, outperforming each baseline. Among pipeline-aware variants, RankFusion achieves higher expected gains but at the cost of greater variance, while CAPA emerges as the most consistent strategy, outperforming standard UHerding on three out of four datasets.

Problem

• Active Learning
• Cascaded Object Detection
• Table Extraction
• Annotation Efficiency
• Inter-stage Dependency

Innovation

• Active Learning
• Cascaded Object Detection
• Uncertainty Herding
• Table Structure Recognition
• Pipeline-aware Sampling