Instance-Optimized String Fingerprints

2025-07-14

AI Summary

Cloud data warehouses exhibit poor efficiency for LIKE queries on string columns under text-intensive workloads; existing dictionary encoding and prefix pruning techniques fail to balance accuracy and overhead. This paper proposes a workload-aware, instance-optimized string fingerprinting technique: it is the first to formulate fingerprint generation as a mixed-integer programming (MIP) problem, enabling lightweight approximate indexing to accelerate columnar LIKE matching. The method generalizes to unseen predicates without retraining. Experiments on DuckDB v1.3 with the IMDb dataset demonstrate up to 1.36× speedup in column scan performance and substantial reductions in CPU and I/O overhead. The core contribution is an MIP-driven, generalizable fingerprint optimization framework—establishing a new paradigm for efficient text-column querying in cloud data warehouses.

Technology Category

• Data Mining & Knowledge Management: Intelligent Query Processing
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

Recent research found that cloud data warehouses are text-heavy. However, their capabilities for efficiently processing string columns remain limited, relying primarily on techniques like dictionary encoding and prefix-based partition pruning. In recent work, we introduced string fingerprints - a lightweight secondary index structure designed to approximate LIKE predicates, albeit with false positives. This approach is particularly compelling for columnar query engines, where fingerprints can help reduce both compute and I/O overhead. We show that string fingerprints can be optimized for specific workloads using mixed-integer optimization, and that they can generalize to unseen table predicates. On an IMDb column evaluated in DuckDB v1.3, this yields table-scan speedups of up to 1.36$ imes$.

Problem

• Optimize string processing in cloud data warehouses
• Improve LIKE predicate approximation with fingerprints
• Enhance columnar query engine performance via workload-specific fingerprints

Innovation

• Lightweight secondary index for LIKE predicates
• Workload-optimized via mixed-integer optimization
• Reduces compute and I/O in columnar engines