🤖 AI Summary
To address low semantic recall accuracy and high latency in large-enterprise knowledge base retrieval, this paper proposes a retrieval-augmented framework leveraging large language models (LLMs) to dynamically generate structured metadata. Methodologically, it integrates semantic, recursive, and naive chunking strategies; enhances vector representations via TF-IDF–weighted embeddings and prefix fusion; and employs a cross-encoder for re-ranking to construct high-quality ground-truth labels. A novel metadata consistency metric is introduced to rigorously evaluate retrieval effectiveness. Experimental results demonstrate that recursive chunking combined with TF-IDF–weighted embeddings achieves 82.5% precision, while naive chunking with prefix fusion attains a Hit Rate@10 of 0.925—both significantly outperforming content-only baselines. The framework delivers superior accuracy, low latency, and strong scalability, making it suitable for enterprise-scale knowledge retrieval.
📝 Abstract
In enterprise settings, efficiently retrieving relevant information from large and complex knowledge bases is essential for operational productivity and informed decision-making. This research presents a systematic framework for metadata enrichment using large language models (LLMs) to enhance document retrieval in Retrieval-Augmented Generation (RAG) systems. Our approach employs a comprehensive, structured pipeline that dynamically generates meaningful metadata for document segments, substantially improving their semantic representations and retrieval accuracy. Through extensive experiments, we compare three chunking strategies-semantic, recursive, and naive-and evaluate their effectiveness when combined with advanced embedding techniques. The results demonstrate that metadata-enriched approaches consistently outperform content-only baselines, with recursive chunking paired with TF-IDF weighted embeddings yielding an 82.5% precision rate compared to 73.3% for semantic content-only approaches. The naive chunking strategy with prefix-fusion achieved the highest Hit Rate@10 of 0.925. Our evaluation employs cross-encoder reranking for ground truth generation, enabling rigorous assessment via Hit Rate and Metadata Consistency metrics. These findings confirm that metadata enrichment enhances vector clustering quality while reducing retrieval latency, making it a key optimization for RAG systems across knowledge domains. This work offers practical insights for deploying high-performance, scalable document retrieval solutions in enterprise settings, demonstrating that metadata enrichment is a powerful approach for enhancing RAG effectiveness.