Reliable Answers for Recurring Questions: Boosting Text-to-SQL Accuracy with Template Constrained Decoding

2026-04-30

AI Summary

This work addresses the challenge that large language models often generate syntactically invalid or semantically inconsistent SQL queries when confronted with complex or unseen database schemas in Text-to-SQL tasks, undermining deployment reliability. To mitigate this issue, the authors propose the Template-Constrained Decoding (TeCoD) framework, which extracts reusable query templates from historical natural language–SQL pairs and leverages a fine-tuned natural language inference model for efficient template matching. TeCoD further integrates grammar-constrained decoding with a partitioned generation strategy to guarantee syntactic validity and computational efficiency of the output. Experimental results demonstrate that the proposed method improves execution accuracy by 36% over in-context learning (ICL) on matched queries and reduces inference latency by a factor of 2.2.

Abstract

Large language models (LLMs) have revolutionized Text-to-SQL generation, allowing users to query structured data using natural language with growing ease. Yet, real-world deployment remains challenging, especially in complex or unseen schemas, due to inconsistent accuracy and the risk of generating invalid SQL. We introduce Template Constrained Decoding (TeCoD), a system that addresses these limitations by harnessing the recurrence of query patterns in labeled workloads. TeCoD converts historical NL-SQL pairs into reusable templates and introduces a robust template selection module that uses a fine-tuned natural language inference model to match or reject queries efficiently. Once the template is selected, TeCoD enforces it during SQL generation through grammar-constrained decoding, implemented via a novel partitioned strategy that ensures both syntactic validity and efficiency. Together, these components yield up to 36% higher execution accuracy than in-context learning (ICL) and 2.2x lower latency on matched queries.

Problem

• Text-to-SQL
• large language models
• SQL generation
• schema complexity
• query accuracy

Innovation

• Template Constrained Decoding
• Text-to-SQL
• Grammar-Constrained Decoding
• Natural Language Inference
• Query Templates