Spatial Information Integration in Small Language Models for Document Layout Generation and Classification

2025-01-09

AI Summary

To address the scarcity of real-world data and the difficulty small language models (SLMs) face in effectively modeling spatial structures for semi-structured document layout understanding, this paper proposes a spatial information integration framework tailored for SLMs. Methodologically: (1) it introduces a coordinate-based synthetic layout generation mechanism to alleviate annotation scarcity; (2) it designs a bounding-box-aware text encoder to enable lightweight joint modeling of spatial and semantic information. Contributions include: (i) the first spatial information fusion paradigm specifically customized for SLMs; and (ii) empirical validation that synthetic layout data significantly improves downstream performance—achieving superior layout generation metrics compared to LayoutTransformer and substantially boosting multi-class document classification accuracy through bounding-box integration.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Document layout understanding is a field of study that analyzes the spatial arrangement of information in a document hoping to understand its structure and layout. Models such as LayoutLM (and its subsequent iterations) can understand semi-structured documents with SotA results; however, the lack of open semi-structured data is a limitation in itself. While semi-structured data is common in everyday life (balance sheets, purchase orders, receipts), there is a lack of public datasets for training machine learning models for this type of document. In this investigation we propose a method to generate new, synthetic, layout information that can help overcoming this data shortage. According to our results, the proposed method performs better than LayoutTransformer, another popular layout generation method. We also show that, in some scenarios, text classification can improve when supported by bounding box information.

Problem

• Document Layout Understanding
• Spatial Information Integration
• Semi-Structured Document Data Scarcity

Innovation

• Synthetic Layout Generation
• Data Scarcity Overcoming
• Bounding Box Information Integration