athena

Existing serverless data analytics systems rely on manual tuning or static execution plans, making it infeasible to automatically identify Pareto-optimal query plans that jointly minimize cost and latency amid an exponential space of feasible plans. This paper proposes the first end-to-end, serverless-native query optimization framework. It introduces a state-space pruning strategy and a novel search algorithm to automatically discover Pareto-optimal plans without human intervention. The framework integrates a lightweight cost model, an adaptive planner, and a FaaS-native execution engine to enable real-time plan evaluation and execution for complex queries. Experiments across diverse real-world workloads demonstrate that our system reduces total cost by 37% on average and improves query latency by 2.1× over AWS Athena. To the best of our knowledge, this is the first approach to achieve automated, provably Pareto-optimal query optimization in serverless environments.

To address low resource utilization, poor long-term operational stability, and high operational costs of cloud-based data processing systems under unpredictable workloads, this paper proposes the first end-to-end fully serverless SQL query processor. The system leverages cloud functions (e.g., AWS Lambda) and serverless storage (e.g., Amazon S3) to achieve full-stack serverless compute and storage. It introduces three key innovations: (1) adaptive execution scheduling, (2) cost-aware resource orchestration, and (3) dynamic query decomposition with elastic workflow orchestration and operator-level cost-driven deployment—overcoming performance and cost bottlenecks inherent in function-as-a-service for long-running analytical workloads. Evaluated on the TB-scale TPC-H benchmark, the system matches mainstream cloud data platforms in both query latency and cost per query. This work provides the first empirical validation of the feasibility and engineering competitiveness of fully serverless architectures for complex analytical workloads.

This work addresses the CPU computational bottleneck encountered by Presto in large-scale analytical queries by introducing the first deep integration of GPU execution into the Presto framework. The authors propose an end-to-end query processing architecture that retains data in GPU memory throughout the execution pipeline. By redesigning the execution engine, they enable GPU-aware scheduling, efficient data transfer from storage to GPU, and intra-operator data exchange directly within GPU memory in distributed environments. The system is built upon NVIDIA’s cuDF C++ library and guided by TPC-H benchmarks for optimization. Experimental results demonstrate that, under standard analytical workloads, the proposed approach achieves up to a 6× improvement in cost-performance ratio compared to native CPU-based Presto. The implementation has been upstreamed into the open-source Presto/Velox engine and deployed in production.

To address the high communication overhead and poor scalability of serverless architectures in machine learning–intensive, data-heavy workloads, this paper proposes a high-performance computing (HPC)-inspired serverless framework. Methodologically, it introduces a NAT-traversal direct communication mechanism based on TCP hole punching, implements a lightweight serverless communicator, and integrates the Cylon distributed dataframe library with an FMI-inspired heuristic communication scheduling model. This design enables decentralized, low-latency, high-throughput distributed data processing within cloud-native environments—without centralized coordination. Experimental results demonstrate that the framework achieves over 99% end-to-end performance improvement compared to conventional serverless approaches. Its strong scaling efficiency closely matches that of EC2 instances and dedicated HPC clusters. Notably, this work is the first to achieve near-HPC communication efficiency and scalability in a serverless setting.

Existing metadata catalogs (e.g., Hive Metastore, Iceberg, Delta Lake) face fundamental limitations in concurrent read/write throughput, strong consistency guarantees, and expressive query capabilities for hyperscale data systems. TreeCat addresses these challenges by introducing the first high-performance, purpose-built metadata catalog engine. It employs a hierarchical data model and a path-based query language; implements MVOCC—a multi-version optimistic concurrency control protocol ensuring serializable isolation; and pioneers an associative scan execution mechanism to accelerate metadata traversal and filtering. Its custom storage format is co-designed for efficient range queries and fine-grained version management. Experimental evaluation demonstrates that TreeCat delivers strict consistency under high concurrency while achieving up to 8.2× higher throughput for range queries compared to state-of-the-art systems, significantly outperforming existing solutions across key metadata management workloads.

This work addresses the challenges of executing temporal OLAP queries in serverless architectures, where computation partitioning often leads to missing state and incorrect aggregations—particularly compromising correctness for non-trivial aggregations such as duration-weighted sums or cumulative threshold computations. To resolve this, the authors propose a novel approach that replicates overlapping time intervals across partitions during a pre-query phase and merges partial results at query time according to operator semantics, guaranteeing output identical to that of single-process execution. This is the first solution to simultaneously ensure both temporal state completeness and correctness of complex aggregations in serverless environments. Experimental evaluation on real-world datasets demonstrates up to 82% lower latency and 84% reduced cost compared to SQL baselines, while maintaining generality and high efficiency.

Serverless computing has matured into an effective execution model for edge cloud environments, enabling function level decomposition, demand driven scaling, and workflow execution across stable, well provisioned infrastructure. This success motivates extending it to the edge cloud space continuum, where Low Earth Orbit (LEO) constellations are increasingly explored as distributed compute substrates. However, existing serverless orchestration is not directly applicable in this setting, where LEO systems impose time varying contact graphs, intermittent link availability, and strict feasibility constraints on energy, memory, communication, and operational cost. This article identifies ten broken assumptions in existing serverless orchestration and organizes them into three core challenges: spatiotemporal execution over dynamic graphs, constraint aware function placement and scaling, and correctness and progress under decentralized and delayed state. It then proposes an architecture that enables robust and efficient serverless execution across the continuum, grounded in these challenges and demonstrated through a representative flood response use case.

This study addresses the lack of systematic performance evaluation of Arrow-native analytics engines within Intel SGX2 trusted execution environments, particularly under large-scale data and realistic cloud storage conditions. For the first time, we execute Polars—an Arrow-native DataFrame library—inside an SGX2 enclave using Gramine, conducting end-to-end measurements of query computation and data loading overheads based on the TPC-H SF30 benchmark and Azure Blob Storage. We further compare lazy and eager execution modes. Our experiments reveal that overall overhead remains stable between 1.49× and 1.56×; computation overhead decreases with larger datasets while data loading overhead increases significantly. Lazy execution outperforms eager execution by more than 2.25× and effectively avoids out-of-memory errors under high memory pressure, highlighting the critical impact of API design choices on both performance and resource utilization.

In data lake queries, decoding Parquet files from remote storage can account for up to 46% of runtime overhead. To address this, this work proposes a cloud-oriented SmartNIC architecture that, for the first time, leverages SmartNICs to directly process pre-filtered columnar data. By offloading Parquet decoding and pushing down query operators onto the network data path, the design enables hardware-accelerated query execution. Integrated with DuckDB, the approach substantially reduces CPU resource consumption. Experimental results demonstrate that after SmartNIC-based pre-filtering, a modest CPU configuration achieves query throughput comparable to that of conventional systems relying on significantly more powerful host processors.