Learned Adaptive Indexing

2025-08-05

AI Summary

To address the high maintenance overhead of traditional indexes under dynamic query workloads and continuous data updates, and the lack of real-time adaptability in existing learned indexes, this paper proposes an Online Learned Adaptive Index (OLAI). OLAI is the first approach to embed machine learning models directly into the index construction pipeline, enabling incremental index building and updating during query execution. It jointly models query workload sequences and performs short-term prediction to drive dynamic structural evolution and localized optimization of the index. Compared with state-of-the-art adaptive indexing schemes, OLAI achieves 1.2×–5.6× higher query throughput across diverse dynamic workloads, significantly reduces response latency, and enhances system adaptability. Its core innovation lies in establishing a closed-loop adaptive mechanism where “queries serve as training signals and optimization triggers,” thereby unifying indexing, learning, and adaptation in real time.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Online Learning & Bandits
• Data Mining & Knowledge Management: Intelligent Query Processing

Application Category

• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Indexes can significantly improve search performance in relational databases. However, if the query workload changes frequently or new data updates occur continuously, it may not be worthwhile to build a conventional index upfront for query processing. Adaptive indexing is a technique in which an index gets built on the fly as a byproduct of query processing. In recent years, research in database indexing has taken a new direction where machine learning models are employed for the purpose of indexing. These indexes, known as learned indexes, can be more efficient compared to traditional indexes such as B+-tree in terms of memory footprints and query performance. However, a learned index has to be constructed upfront and requires training the model in advance, which becomes a challenge in dynamic situations when workload changes frequently. To the best of our knowledge, no learned indexes exist yet for adaptive indexing. We propose a novel learned approach for adaptive indexing. It is built on the fly as queries are submitted and utilizes learned models for indexing data. To enhance query performance, we employ a query workload prediction technique that makes future workload projection based on past workload data. We have evaluated our learned adaptive indexing approach against existing adaptive indexes for various query workloads. Our results show that our approach performs better than others in most cases, offering 1.2x - 5.6x improvement in query performance.

Problem

• Adaptive indexing for dynamic query workloads
• Learned indexes without upfront model training
• Improving query performance with workload prediction

Innovation

• Learned models for on-the-fly indexing
• Workload prediction enhances query performance
• Combines adaptive and learned indexing techniques