Research Challenges in Relational Database Management Systems for LLM Queries

2025-08-28

AI Summary

Existing open-source solutions face critical bottlenecks—limited functionality, uncontrolled structured output, high resource overhead, and lack of query plan co-optimization—when deeply integrating large language models (LLMs) into relational database management systems (RDBMSs). To address these, we propose a tightly coupled LLM+DBMS architecture: (1) a SQL-native LLM invocation syntax with inline calls, and (2) a structured-output constraint mechanism to ensure predictable, schema-compliant results. We further design an LLM-aware optimizer grounded in query rewriting, enabling resource-aware scheduling and execution-path optimization. We implement our prototype on PostgreSQL and SQLite and evaluate it across five representative LLM-augmented query workloads. Compared to baseline systems, our approach reduces end-to-end latency by 42–68%, improves throughput by 3.1×, and scales effectively under high-concurrency LLM query loads. This work delivers a reusable, system-level solution for native LLM support in databases.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) have become essential for applications such as text summarization, sentiment analysis, and automated question-answering. Recently, LLMs have also been integrated into relational database management systems to enhance querying and support advanced data processing. Companies such as Amazon, Databricks, Google, and Snowflake offer LLM invocation directly within SQL, denoted as LLM queries, to boost data insights. However, open-source solutions currently have limited functionality and poor performance. In this work, we present an early exploration of two open-source systems and one enterprise platform, using five representative queries to expose functional, performance, and scalability limits in today's SQL-invoked LLM integrations. We identify three main issues: enforcing structured outputs, optimizing resource utilization, and improving query planning. We implemented initial solutions and observed improvements in accommodating LLM powered SQL queries. These early gains demonstrate that tighter integration of LLM+DBMS is the key to scalable and efficient processing of LLM queries.

Problem

• Enforcing structured outputs from LLM queries
• Optimizing resource utilization in database systems
• Improving query planning for LLM integrations

Innovation

• Enforcing structured outputs for integration
• Optimizing resource utilization efficiency
• Improving query planning strategies