Are Your LLM-based Text-to-SQL Models Secure? Exploring SQL Injection via Backdoor Attacks

2025-03-07

AI Summary

This work exposes a critical security vulnerability in LLM-based Text-to-SQL models: they are highly susceptible to backdoor attacks, where attackers can poison merely 0.44% of training data to induce the model to generate malicious, executable SQL queries—even on benign inputs. To address this, the authors propose ToxicSQL, the first framework that formalizes SQL injection as a backdoor attack objective. It introduces novel dual-granularity triggers—semantic and character-level—that are stealthy yet effective. ToxicSQL achieves a high attack success rate of 79.41% while preserving 98.2% accuracy on clean inputs, demonstrating strong functional fidelity and concealment. Furthermore, the framework incorporates lightweight, deployable detection and mitigation strategies, significantly enhancing the robustness and security of Text-to-SQL models in real-world database environments.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Large language models (LLMs) have shown state-of-the-art results in translating natural language questions into SQL queries (Text-to-SQL), a long-standing challenge within the database community. However, security concerns remain largely unexplored, particularly the threat of backdoor attacks, which can introduce malicious behaviors into models through fine-tuning with poisoned datasets. In this work, we systematically investigate the vulnerabilities of LLM-based Text-to-SQL models and present ToxicSQL, a novel backdoor attack framework. Our approach leverages stealthy {semantic and character-level triggers} to make backdoors difficult to detect and remove, ensuring that malicious behaviors remain covert while maintaining high model accuracy on benign inputs. Furthermore, we propose leveraging SQL injection payloads as backdoor targets, enabling the generation of malicious yet executable SQL queries, which pose severe security and privacy risks in language model-based SQL development. We demonstrate that injecting only 0.44% of poisoned data can result in an attack success rate of 79.41%, posing a significant risk to database security. Additionally, we propose detection and mitigation strategies to enhance model reliability. Our findings highlight the urgent need for security-aware Text-to-SQL development, emphasizing the importance of robust defenses against backdoor threats.

Problem

• Explores SQL injection vulnerabilities in LLM-based Text-to-SQL models.
• Introduces ToxicSQL, a backdoor attack framework using stealthy triggers.
• Proposes detection and mitigation strategies to enhance model security.

Innovation

• ToxicSQL: novel backdoor attack framework
• Stealthy semantic and character-level triggers
• SQL injection payloads as backdoor targets