🤖 AI Summary
This work bridges the gap between theory and practice in query-driven selectivity estimation, relaxing the conventional i.i.d. assumption inherent in PAC learning. We propose a novel modeling framework based on signed measures, establishing—for the first time—theoretical out-of-distribution (OOD) generalization error bounds for selectivity predictors. We provide rigorous learnability analysis and derive two generic strategies to improve OOD generalization. Empirically, our approach significantly enhances prediction accuracy and query latency under OOD workloads while preserving strong in-distribution generalization performance. Comprehensive theoretical analysis and extensive experiments jointly validate the framework’s effectiveness and conceptual novelty.
📝 Abstract
Query-driven machine learning models have emerged as a promising estimation technique for query selectivities. Yet, surprisingly little is known about the efficacy of these techniques from a theoretical perspective, as there exist substantial gaps between practical solutions and state-of-the-art (SOTA) theory based on the Probably Approximately Correct (PAC) learning framework. In this paper, we aim to bridge the gaps between theory and practice. First, we demonstrate that selectivity predictors induced by signed measures are learnable, which relaxes the reliance on probability measures in SOTA theory. More importantly, beyond the PAC learning framework (which only allows us to characterize how the model behaves when both training and test workloads are drawn from the same distribution), we establish, under mild assumptions, that selectivity predictors from this class exhibit favorable out-of-distribution (OOD) generalization error bounds. These theoretical advances provide us with a better understanding of both the in-distribution and OOD generalization capabilities of query-driven selectivity learning, and facilitate the design of two general strategies to improve OOD generalization for existing query-driven selectivity models. We empirically verify that our techniques help query-driven selectivity models generalize significantly better to OOD queries both in terms of prediction accuracy and query latency performance, while maintaining their superior in-distribution generalization performance.