🤖 AI Summary
This work addresses the limitations of generic Retrieval-Augmented Generation (RAG) approaches in enterprise data analytics, which suffer from low retrieval accuracy (Hit@10 of only 19.1%) and misinterpretation of metrics due to semantic gaps, entity ambiguity, schema drift, and misalignment between data assets and their usage contexts. To overcome these challenges, the authors propose a dual-layer architecture: the lower layer constructs a three-tier, dual-purpose knowledge base integrating a knowledge graph with 2,859 nodes and eight-segment scenario annotations; the upper layer introduces a graph-guided retriever (GGR) and a scenario-aware ranker (SAR), enhanced by negative knowledge augmentation and a lightweight closed-loop hot-reload mechanism enabling daily knowledge updates. Evaluated on two hundred-question benchmarks, the approach achieves a Hit@10 of 96.6%, increases knowledge coverage to 77%, and maintains end-to-end latency between 4.84 and 5.33 seconds.
📝 Abstract
Enterprise data analytics agents face two structural failures: generic RAG retrieves the wrong asset (Hit@10=19.1%) and delivers no usage knowledge to prevent metric misinterpretation---stemming from four root causes (C1--C4) ranging from semantic gap and entity ambiguity to schema drift and asset-usage gap. We present a two-layer solution deployed in the commercial advertising data warehouse at Xiaohongshu (5,300+ Hive tables, 14 domains). A three-tier dual-purpose knowledge base (179 documents, eight-section annotation template) serves both retrieval and generation, with a closed-loop refresh pipeline maintaining day-level freshness (one yes/no approval, 30s hot-reload). The Graph-Guided Retriever (GGR) uses a 2,859-node knowledge graph as a candidate gate with intent routing to deliver 71.6x token reduction. The Scene-Aware Ranker (SAR) applies 19-class entity recognition and explicit scenario annotations; negative knowledge alone contributes 25 percentage points of Hit@10 gain. On two 100-question benchmarks, Hit@10 rises from 19.1% to 96.6% (+77.5pp) and knowledge coverage from 56% to 77%, at 4.84--5.33s end-to-end latency.