trino

Existing serverless data analytics systems rely on manual tuning or static execution plans, making it infeasible to automatically identify Pareto-optimal query plans that jointly minimize cost and latency amid an exponential space of feasible plans. This paper proposes the first end-to-end, serverless-native query optimization framework. It introduces a state-space pruning strategy and a novel search algorithm to automatically discover Pareto-optimal plans without human intervention. The framework integrates a lightweight cost model, an adaptive planner, and a FaaS-native execution engine to enable real-time plan evaluation and execution for complex queries. Experiments across diverse real-world workloads demonstrate that our system reduces total cost by 37% on average and improves query latency by 2.1× over AWS Athena. To the best of our knowledge, this is the first approach to achieve automated, provably Pareto-optimal query optimization in serverless environments.

Multi-cloud ETL faces significant challenges, including high cross-cloud data movement costs, incompatibility among heterogeneous SQL engines, complex orchestration, and fragmented security policies. This work presents the first systematic evaluation of predicate pushdown optimization in multi-cloud ETL scenarios and proposes a novel协同 strategy that integrates localized pushdown with data federation to offload transformation logic across heterogeneous query engines such as Amazon Redshift and Google BigQuery. By intelligently pushing computation closer to the data sources while leveraging federated querying capabilities, the approach substantially reduces cross-cloud data transfer, thereby lowering end-to-end execution latency and cost. The proposed paradigm offers an efficient and scalable solution for optimizing multi-cloud data integration workflows.

To address the resource waste and unpredictable query latency caused by static degree of parallelism (DOP) configuration during query execution in cloud-native OLAP systems, this paper proposes Intra-Query Runtime Elasticity (IQRE). IQRE enables dynamic, zero-interruption DOP adjustment at any execution point without halting data processing. It introduces the first delay-constraint-driven “what-if” automated tuning service integrated with a user-friendly interactive interface. Built upon the Presto execution model, IQRE features a lightweight runtime scheduler, an elastic task redistribution protocol, and an online resource-aware optimization framework. Experimental evaluation under realistic workloads demonstrates that IQRE reduces average computational resource consumption by 37% while guaranteeing 100% compliance with end-to-end query latency constraints.

This work addresses the challenges of traditional AI analytics in multi-tenant heterogeneous data systems, including high computational overhead, limited robustness to data drift, and heightened security risks. To overcome these limitations, the paper proposes a deep integration of AI capabilities into the database engine, co-optimizing query processing and model execution while re-engineering transaction management and access control mechanisms to support efficient end-to-end AI lifecycle operations. The authors design a unified optimization and secure execution framework for AI-native databases (AIxDB), which synergistically combines query optimization, distributed scheduling, heterogeneous hardware adaptation, and fine-grained security enforcement. Preliminary experiments demonstrate the architecture’s effectiveness in mitigating data drift and optimizing resource scheduling, offering a foundational technical pathway toward secure, high-performance AI-native database systems.

Existing metadata catalogs (e.g., Hive Metastore, Iceberg, Delta Lake) face fundamental limitations in concurrent read/write throughput, strong consistency guarantees, and expressive query capabilities for hyperscale data systems. TreeCat addresses these challenges by introducing the first high-performance, purpose-built metadata catalog engine. It employs a hierarchical data model and a path-based query language; implements MVOCC—a multi-version optimistic concurrency control protocol ensuring serializable isolation; and pioneers an associative scan execution mechanism to accelerate metadata traversal and filtering. Its custom storage format is co-designed for efficient range queries and fine-grained version management. Experimental evaluation demonstrates that TreeCat delivers strict consistency under high concurrency while achieving up to 8.2× higher throughput for range queries compared to state-of-the-art systems, significantly outperforming existing solutions across key metadata management workloads.

This work addresses the scarcity of real-world SQL workloads and limitations of existing synthetic query generation methods—such as rigid templates, schema hallucination, and simplistic join structures—that hinder effective training of query optimizers. The authors propose SynQL, a rule-based, deterministic SQL synthesis framework that constructs abstract syntax trees by traversing foreign key graphs in database schemas, eschewing probabilistic generation to guarantee syntactic and schema-level validity. SynQL introduces a configuration vector to explicitly control join topology, analytical complexity, and predicate selectivity, while modeling core query components including multi-table joins, projections, aggregations, and range predicates. Workloads generated on TPC-H and IMDb achieve a topological entropy of 1.53 bits; tree-based models trained on this data attain R² ≥ 0.79 on test sets with sub-millisecond inference latency.

This work addresses the CPU computational bottleneck encountered by Presto in large-scale analytical queries by introducing the first deep integration of GPU execution into the Presto framework. The authors propose an end-to-end query processing architecture that retains data in GPU memory throughout the execution pipeline. By redesigning the execution engine, they enable GPU-aware scheduling, efficient data transfer from storage to GPU, and intra-operator data exchange directly within GPU memory in distributed environments. The system is built upon NVIDIA’s cuDF C++ library and guided by TPC-H benchmarks for optimization. Experimental results demonstrate that, under standard analytical workloads, the proposed approach achieves up to a 6× improvement in cost-performance ratio compared to native CPU-based Presto. The implementation has been upstreamed into the open-source Presto/Velox engine and deployed in production.

Accurately estimating slot-time—the computational resource consumption—prior to query execution in cloud data warehouses remains challenging, often leading to budget overruns and inefficient scheduling. This work proposes a pre-execution prediction framework that leverages only observable features such as SQL structure, planner estimates, and query text. It integrates complexity scores, metadata, and TF-IDF/TruncatedSVD-512 text embeddings into a dual-model architecture based on HistGradientBoostingRegressor, augmented with a complexity-aware routing mechanism. Evaluated on 746 out-of-distribution queries, the approach achieves a mean absolute error (MAE) of 1.17, root mean squared error (RMSE) of 4.71, and 74% explained variance. For high-cost queries, it reduces MAE by 30–37% compared to mean/median baselines, substantially improving prediction accuracy and generalization.

This work addresses the challenges of implementing analytical logic in production-grade systems using imperative code, which often leads to code duplication, definition drift, and poor testability. To overcome these issues, the authors propose modeling analytical workflows as typed, declarative directed acyclic graphs (QDAGs), enabling, for the first time, a unified representation of complex analytical pipelines in large-scale production environments. The system efficiently orchestrates multi-step operations through on-demand execution, memoization, pruning, and parallel scheduling, supporting diverse primitives such as Apache Pinot queries, service invocations, SQLite joins, jq transformations, conditional logic, differentially private aggregations, and nested QDAG calls. Deployed across over 100 production use cases at LinkedIn on more than 500 hosts, the system achieves median orchestration overhead of approximately 10 milliseconds and 99th-percentile latency under 50 milliseconds, significantly enhancing reusability, consistency, and testability while meeting interactive latency requirements.

This work addresses the performance limitations of general-purpose relational database management systems (RDBMS) on specific query workloads and the limited adoption of custom accelerators due to high development costs and narrow applicability. The authors propose Tailwind, a framework that seamlessly integrates diverse accelerators into any RDBMS supporting data import/export by leveraging an external query planner. Its key innovation is the introduction of an Abstract Logical Plan (ALP)—a declarative intermediate representation—combined with a neural cost model that automatically predicts acceleration benefits. At runtime, Tailwind transparently rewrites queries to invoke the optimal accelerator without modifying the database kernel. Experiments integrating four accelerators into Amazon Redshift and DuckDB demonstrate an average speedup of 1.38× on TPC-H queries, with peak improvements reaching 29×.