RubikSQL: Lifelong Learning Agentic Knowledge Base as an Industrial NL2SQL System

2025-08-24

AI Summary

Enterprise-level NL2SQL systems face critical challenges in interpreting implicit user intent, adapting to domain-specific terminology, and enabling continuous knowledge evolution. Method: This paper pioneers modeling NL2SQL as a continual lifelong learning task and proposes a multi-agent collaborative framework. It introduces automated, dynamic construction and maintenance of a knowledge base via database profiling and SQL profiling—integrating structured information extraction, rule mining, and chain-of-thought–enhanced reasoning—while decoupling knowledge evolution from SQL generation. Contribution/Results: To advance industrial evaluation, we release RubikBench, the first benchmark targeting complex enterprise queries. Our approach achieves state-of-the-art performance on KaggleDBQA and BIRD Mini-Dev, with substantial improvements in accuracy for multi-hop, nested, and domain-specific queries, as well as enhanced system scalability and maintainability.

Technology Category

• Data Mining & Knowledge Management: Intelligent Query Processing
• Natural Language Processing: Question Answering
• Machine Learning: Life-Long and Continual Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We present RubikSQL, a novel NL2SQL system designed to address key challenges in real-world enterprise-level NL2SQL, such as implicit intents and domain-specific terminology. RubikSQL frames NL2SQL as a lifelong learning task, demanding both Knowledge Base (KB) maintenance and SQL generation. RubikSQL systematically builds and refines its KB through techniques including database profiling, structured information extraction, agentic rule mining, and Chain-of-Thought (CoT)-enhanced SQL profiling. RubikSQL then employs a multi-agent workflow to leverage this curated KB, generating accurate SQLs. RubikSQL achieves SOTA performance on both the KaggleDBQA and BIRD Mini-Dev datasets. Finally, we release the RubikBench benchmark, a new benchmark specifically designed to capture vital traits of industrial NL2SQL scenarios, providing a valuable resource for future research.

Problem

• Addresses implicit intents in enterprise NL2SQL systems
• Handles domain-specific terminology for industrial databases
• Implements lifelong learning for knowledge base maintenance

Innovation

• Lifelong learning Knowledge Base maintenance
• Agentic rule mining for SQL generation
• Chain-of-Thought enhanced SQL profiling