🤖 AI Summary
To address poor generalization of models for visually rich document (VRD) understanding—caused by inconsistent layouts across scans and scarce domain-specific annotations—this paper proposes a lightweight domain-adaptive framework. Our method introduces a novel hierarchical document representation that jointly encodes fine-grained modalities (text and layout) and coarse-grained semantics (semantic meaning and structural relations), coupled with a domain-aware feature alignment mechanism. Furthermore, we employ controllable generation to synthesize high-fidelity VRD data, replacing costly manual annotation and enabling synthetic-data-driven supervised fine-tuning. Evaluated on multiple downstream tasks—including key information extraction and table recognition—the framework achieves state-of-the-art performance using less than 5% of real annotated data per target domain. This demonstrates substantial reduction in cross-domain adaptation cost and establishes a new paradigm for VRD understanding: low-resource, highly generalizable, and robust.
📝 Abstract
Visually-Rich Documents (VRDs), encompassing elements like charts, tables, and references, convey complex information across various fields. However, extracting information from these rich documents is labor-intensive, especially given their inconsistent formats and domain-specific requirements. While pretrained models for VRD Understanding have progressed, their reliance on large, annotated datasets limits scalability. This paper introduces the Domain Adaptive Visually-rich Document Understanding (DAViD) framework, which utilises machine-generated synthetic data for domain adaptation. DAViD integrates fine-grained and coarse-grained document representation learning and employs synthetic annotations to reduce the need for costly manual labelling. By leveraging pretrained models and synthetic data, DAViD achieves competitive performance with minimal annotated datasets. Extensive experiments validate DAViD's effectiveness, demonstrating its ability to efficiently adapt to domain-specific VRDU tasks.