LLM-Based SQL Generation: Prompting, Self-Refinement, and Adaptive Weighted Majority Voting

2026-01-25

AI Summary

This work addresses key challenges in natural language to SQL (NL2SQL) tasks, including query ambiguity, complex schema linking, poor cross-dialect generalization, and insufficient domain knowledge. The authors propose the SSEV pipeline, which integrates self-refinement with a weighted majority voting mechanism, and introduce ReCAPAgent-SQL—a multi-agent collaborative framework that uniquely combines single-agent self-refinement with ensemble voting. This framework incorporates planning, retrieval, critique, and verification mechanisms to support enterprise-grade complex scenarios. Built upon the PET-SQL architecture, the system achieves execution accuracies of 85.5%, 86.4%, and 66.3% on Spider 1.0-Dev, Spider 1.0-Test, and BIRD-Dev, respectively. Furthermore, ReCAPAgent-SQL attains a 31% accuracy on the first 100 queries of Spider 2.0-Lite, substantially improving SQL generation quality under unsupervised conditions.

Technology Category

• Natural Language Processing: Question Answering
• Multiagent Systems: Adversarial Agents
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Text-to-SQL has emerged as a prominent research area, particularly with the rapid advancement of large language models (LLMs). By enabling users to query databases through natural language rather than SQL, this technology significantly lowers the barrier to data analysis. However, generating accurate SQL from natural language remains challenging due to ambiguity in user queries, the complexity of schema linking, limited generalization across SQL dialects, and the need for domain-specific understanding. In this study, we propose a Single-Agent Self-Refinement with Ensemble Voting (SSEV) pipeline built on PET-SQL that operates without ground-truth data, integrating self-refinement with Weighted Majority Voting (WMV) and its randomized variant (RWMA). Experimental results show that the SSEV achieves competitive performance across multiple benchmarks, attaining execution accuracies of 85.5% on Spider 1.0-Dev, 86.4% on Spider 1.0-Test, and 66.3% on BIRD-Dev. Building on insights from the SSEV pipeline, we further propose ReCAPAgent-SQL (Refinement-Critique-Act-Plan agent-based SQL framework) to address the growing complexity of enterprise databases and real-world Text-to-SQL tasks. The framework integrates multiple specialized agents for planning, external knowledge retrieval, critique, action generation, self-refinement, schema linking, and result validation, enabling iterative refinement of SQL predictions through agent collaboration. ReCAPAgent-SQL's WMA results achieve 31% execution accuracy on the first 100 queries of Spider 2.0-Lite, demonstrating significant improvements in handling real-world enterprise scenarios. Overall, our work facilitates the deployment of scalable Text-to-SQL systems in practical settings, supporting better data-driven decision-making at lower cost and with greater efficiency.

Problem

• Text-to-SQL
• natural language ambiguity
• schema linking
• SQL dialect generalization
• domain-specific understanding

Innovation

• Self-Refinement
• Weighted Majority Voting
• Multi-Agent Collaboration
• Text-to-SQL
• Schema Linking