R-Bot: An LLM-based Query Rewrite System

2024-12-02

arXiv.org

AI Summary

Traditional SQL query rewriting approaches—whether heuristic- or learning-based—suffer from limited accuracy and robustness, while direct LLM invocation (e.g., GPT-4) often yields hallucinated outputs. To address these challenges, this paper proposes the first LLM-augmented framework specifically designed for SQL query rewriting. Our method introduces three key innovations: (1) a multi-source rewriting evidence generation pipeline that jointly leverages syntactic structure, execution feedback, and semantic similarity; (2) a syntax–semantics hybrid retrieval mechanism to enhance context relevance; and (3) a self-reflective, stepwise LLM reasoning and self-verification paradigm. Evaluated across multiple mainstream benchmarks, our approach significantly outperforms state-of-the-art methods, achieving 12.6–28.3% higher rewriting accuracy and a 19.4% improvement in execution success rate, while effectively mitigating hallucination—demonstrating both high precision and strong robustness.

Technology Category

• Data Mining & Knowledge Management: Intelligent Query Processing
• Search and Optimization: Learning to Search
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Query rewrite is essential for optimizing SQL queries to improve their execution efficiency without changing their results. Traditionally, this task has been tackled through heuristic and learning-based methods, each with its limitations in terms of inferior quality and low robustness. Recent advancements in LLMs offer a new paradigm by leveraging their superior natural language and code comprehension abilities. Despite their potential, directly applying LLMs like GPT-4 has faced challenges due to problems such as hallucinations, where the model might generate inaccurate or irrelevant results. To address this, we propose R-Bot, an LLM-based query rewrite system with a systematic approach. We first design a multi-source rewrite evidence preparation pipeline to generate query rewrite evidences for guiding LLMs to avoid hallucinations. We then propose a hybrid structure-semantics retrieval method that combines structural and semantic analysis to retrieve the most relevant rewrite evidences for effectively answering an online query. We next propose a step-by-step LLM rewrite method that iteratively leverages the retrieved evidences to select and arrange rewrite rules with self-reflection. We conduct comprehensive experiments on widely used benchmarks, and demonstrate the superior performance of our system, R-Bot, surpassing state-of-the-art query rewrite methods.

Problem

• Optimizing SQL queries without altering results
• Overcoming limitations of heuristic and learning-based methods
• Reducing hallucinations in LLM-based query rewriting

Innovation

• Multi-source rewrite evidence pipeline prevents hallucinations
• Hybrid structure-semantics retrieval enhances relevance
• Step-by-step LLM rewrite with self-reflection