FS-DAG: Few Shot Domain Adapting Graph Networks for Visually Rich Document Understanding

2025-05-22

AI Summary

Addressing key challenges in few-shot visual-rich document understanding (VRDU)—including poor cross-domain transferability, low robustness to OCR noise, and constrained deployment resources—this paper proposes a modular few-shot domain-adaptive graph network architecture. The method supports dynamic switching between language- and vision-based backbones, and integrates graph neural networks, multimodal feature alignment, domain-adaptive knowledge distillation, and OCR-robust encoding. With fewer than 90M parameters, it achieves both lightweight design and strong generalization. On information extraction tasks, the model significantly accelerates convergence and improves accuracy, consistently outperforming existing state-of-the-art approaches. It provides a scalable, highly robust solution to real-world issues such as OCR errors, spelling variants, and domain shift—enabling effective adaptation under limited supervision and noisy inputs.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In this work, we propose Few Shot Domain Adapting Graph (FS-DAG), a scalable and efficient model architecture for visually rich document understanding (VRDU) in few-shot settings. FS-DAG leverages domain-specific and language/vision specific backbones within a modular framework to adapt to diverse document types with minimal data. The model is robust to practical challenges such as handling OCR errors, misspellings, and domain shifts, which are critical in real-world deployments. FS-DAG is highly performant with less than 90M parameters, making it well-suited for complex real-world applications for Information Extraction (IE) tasks where computational resources are limited. We demonstrate FS-DAG's capability through extensive experiments for information extraction task, showing significant improvements in convergence speed and performance compared to state-of-the-art methods. Additionally, this work highlights the ongoing progress in developing smaller, more efficient models that do not compromise on performance. Code : https://github.com/oracle-samples/fs-dag

Problem

• Adapts to diverse document types with minimal data
• Handles OCR errors, misspellings, and domain shifts
• Efficient Information Extraction with fewer parameters

Innovation

• Modular framework with domain-specific backbones
• Robust to OCR errors and domain shifts
• Efficient sub-90M parameter model