DriftBench: Defining and Generating Data and Query Workload Drift for Benchmarking

2025-10-12

AI Summary

Existing database benchmarking tools lack mechanisms to model data and query workload drift, hindering robustness evaluation of caching, cardinality estimation, indexing, and query optimization under dynamic conditions. To address this, we propose the first unified taxonomy of data and workload drift—spanning both academic and industrial scenarios—along with standardized formal definitions. We further design DriftBench, a lightweight, extensible benchmarking framework that supports rule-based and statistical-model-driven generation of diverse drift types, as well as customizable drift injection and replay. Through three case studies—data drift analysis, workload drift characterization, and drift-aware cardinality estimation—we empirically validate the taxonomy’s expressiveness and the framework’s practicality. DriftBench enables systematic, reproducible assessment of database components under evolving workloads, establishing a new paradigm for evaluating and optimizing database systems in dynamic environments.

Technology Category

• Data Mining & Knowledge Management: Data Stream Mining
• Search and Optimization: Distributed Search
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Data and workload drift are key to evaluating database components such as caching, cardinality estimation, indexing, and query optimization. Yet, existing benchmarks are static, offering little to no support for modeling drift. This limitation stems from the lack of clear definitions and tools for generating data and workload drift. Motivated by this gap, we propose a unified taxonomy for data and workload drift, grounded in observations from both academia and industry. Building on this foundation, we introduce DriftBench, a lightweight and extensible framework for generating data and workload drift in benchmark inputs. Together, the taxonomy and DriftBench provide a standardized vocabulary and mechanism for modeling and generating drift in benchmarking. We demonstrate their effectiveness through case studies involving data drift, workload drift, and drift-aware cardinality estimation.

Problem

• Defining unified taxonomy for data and workload drift
• Generating realistic data and query workload drift
• Enabling drift-aware benchmarking for database components

Innovation

• Proposes unified taxonomy for data and workload drift
• Introduces DriftBench framework for generating drift
• Provides standardized vocabulary and drift mechanism