🤖 AI Summary
The impact of database normalization levels (1NF–3NF) on NL2SQL performance remains poorly understood, hindering principled database design for natural-language-driven query systems.
Method: We systematically evaluate eight large language models across synthetic and real-world datasets spanning varying normalization levels, measuring SQL generation accuracy under zero-shot and few-shot settings.
Contribution/Results: Our empirical study is the first to demonstrate that denormalized schemas improve accuracy for simple retrieval queries, whereas normalized schemas significantly enhance performance on aggregation queries—despite increasing JOIN-related errors. Crucially, few-shot examples effectively mitigate such JOIN errors induced by normalization. Based on these findings, we propose a “query-type–driven adaptive schema selection” strategy, offering actionable guidelines for co-designing database schemas and deploying NL2SQL models. This work bridges database theory and NL2SQL practice, enabling schema-aware model optimization.
📝 Abstract
Schema design, particularly normalization, is a critical yet often overlooked factor in natural language to SQL (NL2SQL) systems. Most prior research evaluates models on fixed schemas, overlooking the influence of design on performance. We present the first systematic study of schema normalization's impact, evaluating eight leading large language models on synthetic and real-world datasets with varied normalization levels. We construct controlled synthetic datasets with formal normalization (1NF-3NF) and real academic paper datasets with practical schemes. Our results show that denormalized schemas offer high accuracy on simple retrieval queries, even with cost-effective models in zero-shot settings. In contrast, normalized schemas (2NF/3NF) introduce challenges such as errors in base table selection and join type prediction; however, these issues are substantially mitigated by providing few-shot examples. For aggregation queries, normalized schemas yielded better performance, mainly due to their robustness against the data duplication and NULL value issues that cause errors in denormalized schemas. These findings suggest that the optimal schema design for NL2SQL applications depends on the types of queries to be supported. Our study demonstrates the importance of considering schema design when developing NL2SQL interfaces and integrating adaptive schema selection for real-world scenarios.