SQLord: A Robust Enterprise Text-to-SQL Solution via Reverse Data Generation and Workflow Decomposition

2025-07-14

AI Summary

Addressing the dual challenges of adapting to complex business logic and scarcity of domain-specific data in text-to-SQL tasks, this paper proposes a synergistic framework integrating reverse data generation and workflow decomposition. First, it employs reverse SQL-to-text generation to automatically construct high-quality, human-annotation-free training data tailored to target domains. Second, it decomposes intricate queries into verifiable subtasks to enhance model reasoning robustness. Furthermore, we introduce GPT-Judge—a multidimensional evaluation framework that jointly assesses execution accuracy (EXE), query-SQL semantic alignment (QSE), and SQL structural equivalence (SSE), eliminating reliance on ground-truth annotations or executable databases. Offline experiments demonstrate significant improvements over state-of-the-art baselines. The framework has been deployed online at a leading global B2B e-commerce platform, achieving stable end-to-end accuracy exceeding 90% across diverse business scenarios.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Data Mining & Knowledge Management: Intelligent Query Processing
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Transforming natural language into SQL queries (NL2SQL) is crucial for data-driven business applications. Existing frameworks, trained on open-source datasets, struggle with complex business logic and lack domain-specific data for fine-tuning. Additionally, evaluation methods often require annotated data and executable database environments, which are scarce in real-world scenarios. To address these challenges, we propose SQLord, an enterprise-level NL2SQL framework. First, SQLord introduces a data reverse generation approach to convert raw SQL statements into annotated data for supervised fine-tuning (SFT). Second, it proposes a decomposition method for complex queries using an automated workflow generator. Additionally, SQLord features a comprehensive GPT-Judge evaluation framework, including Execution Evaluation (EXE), Query-SQL Evaluation (QSE), and SQL-SQL Evaluation (SSE), tailored to diverse scenarios. Offline tests significantly outperform state of the art baselines, and online accuracy consistently exceeds 90, highlighting SQLord's advantages and effectiveness in complex real world scenarios. SQLord has been successfully applied across multiple scenarios on the world's largest B2B e-commerce platform.

Problem

• Overcoming lack of domain-specific data for NL2SQL fine-tuning
• Handling complex business logic in text-to-SQL conversion
• Evaluating NL2SQL without annotated data or executable databases

Innovation

• Reverse data generation for supervised fine-tuning
• Workflow decomposition for complex queries
• GPT-Judge framework for multi-scenario evaluation