RELOAD: A Robust and Efficient Learned Query Optimizer for Database Systems

2026-04-16

AI Summary

Existing reinforcement learning (RL)-based query optimizers suffer from unstable performance, severe degradation, and slow convergence in single-query settings, hindering their practical deployment. This work proposes RELOAD, the first RL-based query optimizer that simultaneously achieves high robustness and high training efficiency. By incorporating stability constraints and an efficient policy learning mechanism, RELOAD effectively suppresses performance fluctuations and accelerates convergence to expert-level plan quality. Experimental results on the JOB, TPC-DS, and SSB benchmarks demonstrate that RELOAD improves robustness by up to 2.4× and training efficiency by up to 3.1× compared to the current state-of-the-art RL-based optimizers.

Technology Category

• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Recent advances in query optimization have shifted from traditional rule-based and cost-based techniques towards machine learning-driven approaches. Among these, reinforcement learning (RL) has attracted significant attention due to its ability to optimize long-term performance by learning policies over query planning. However, existing RL-based query optimizers often exhibit unstable performance at the level of individual queries, including severe performance regressions, and require prolonged training to reach the plan quality of expert, cost-based optimizers. These shortcomings make learned query optimizers difficult to deploy in practice and remain a major barrier to their adoption in production database systems. To address these challenges, we present RELOAD, a robust and efficient learned query optimizer for database systems. RELOAD focuses on (i) robustness, by minimizing query-level performance regressions and ensuring consistent optimization behavior across executions, and (ii) efficiency, by accelerating convergence to expert-level plan quality. Through extensive experiments on standard benchmarks, including Join Order Benchmark, TPC-DS, and Star Schema Benchmark, RELOAD demonstrates up to 2.4x higher robustness and 3.1x greater efficiency compared to state-of-the-art RL-based query optimization techniques.

Problem

• query optimization
• reinforcement learning
• performance regression
• training efficiency
• robustness

Innovation

• learned query optimization
• reinforcement learning
• robustness
• convergence efficiency
• database systems