🤖 AI Summary
This study addresses the limitations of current retrieval-augmented generation (RAG) systems in handling complex engineering documents from the oil and gas industry—particularly multimodal,图文混排 P&ID diagrams—where suboptimal chunking strategies hinder performance. For the first time in a real-world oil and gas documentation setting, the authors systematically evaluate four chunking approaches: fixed sliding window, recursive, semantic breakpoint, and structure-aware chunking, comparing their retrieval effectiveness and computational overhead within an end-to-end RAG framework. Results demonstrate that structure-aware chunking significantly outperforms other methods by preserving logical document structure, achieving superior top-K retrieval metrics while incurring lower computational costs. Moreover, all text-only chunking strategies exhibit fundamental limitations when applied to multimodal documents like P&IDs, underscoring the necessity of integrating multimodal information for effective retrieval.
📝 Abstract
Retrieval-Augmented Generation (RAG) has emerged as a framework to address the constraints of Large Language Models (LLMs). Yet, its effectiveness fundamentally hinges on document chunking - an often-overlooked determinant of its quality. This paper presents an empirical study quantifying performance differences across four chunking strategies: fixed-size sliding window, recursive, breakpoint-based semantic, and structure-aware. We evaluated these methods using a proprietary corpus of oil and gas enterprise documents, including text-heavy manuals, table-heavy specifications, and piping and instrumentation diagrams (P and IDs). Our findings show that structure-aware chunking yields higher overall retrieval effectiveness, particularly in top-K metrics, and incurs significantly lower computational costs than semantic or baseline strategies. Crucially, all four methods demonstrated limited effectiveness on P and IDs, underscoring a core limitation of purely text-based RAG within visually and spatially encoded documents. We conclude that while explicit structure preservation is essential for specialised domains, future work must integrate multimodal models to overcome current limitations.