Dialect-Agnostic SQL Parsing via LLM-Based Segmentation

2026-03-17

AI Summary

Existing SQL analysis tools struggle to handle dialect-specific syntax, while large language models (LLMs) often falter in direct SQL parsing due to insufficient understanding of hierarchical structure and susceptibility to hallucination. To address these limitations, this work proposes SQLFlex, a novel framework that reformulates SQL parsing as a sequential segmentation task well-suited to LLM capabilities. SQLFlex integrates context-free grammars with clause- and expression-level segmentation strategies and incorporates a multi-stage validation mechanism to enhance robustness and reliability. Experimental results demonstrate that SQLFlex achieves parsing success rates of 91.55%–100% across eight SQL dialects, improves the F1 score for SQL linting by 63.68% over SQLFluff, and accelerates test case minimization by up to 10× compared to SQLess, substantially outperforming current baseline approaches.

Technology Category

• Natural Language Processing: Syntax — Tagging, Chunking & Parsing
• Data Mining & Knowledge Management: Intelligent Query Processing
• Search and Optimization: Evaluation and Analysis

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

SQL is a widely adopted language for querying data, which has led to the development of various SQL analysis and rewriting tools. However, due to the diversity of SQL dialects, such tools often fail when encountering unrecognized dialect-specific syntax. While Large Language Models (LLMs) have shown promise in understanding SQL queries, their inherent limitations in handling hierarchical structures and hallucination risks limit their direct applicability in parsing. To address these limitations, we propose SQLFlex, a novel query rewriting framework that integrates grammar-based parsing with LLM-based segmentation to parse diverse SQL dialects robustly. Our core idea is to decompose hierarchical parsing to sequential segmentation tasks, which better aligns with the strength of LLMs and improves output reliability through validation checks. Specifically, SQLFlex uses clause-level segmentation and expression-level segmentation as two strategies that decompose elements on different levels of a query. We extensively evaluated SQLFlex on both real-world use cases and in a standalone evaluation. In SQL linting, SQLFlex outperforms SQLFluff in ANSI mode by 63.68% in F1 score while matching its dialect-specific mode performance. In test-case reduction, SQLFlex outperforms SQLess by up to 10 times in simplification rate. In the standalone evaluation, it parses 91.55% to 100% of queries across eight distinct dialects, outperforming all baseline parsers. We believe SQLFlex can serve as a foundation for many query analysis and rewriting use cases.

Problem

• SQL parsing
• SQL dialects
• Large Language Models
• hierarchical structure
• hallucination

Innovation

• dialect-agnostic SQL parsing
• LLM-based segmentation
• hierarchical parsing decomposition
• clause-level segmentation
• expression-level segmentation