data partitioning

Existing cloud databases simplify scaling as either horizontal or vertical single-dimensional decisions, hindering joint optimization of performance, cost, and coordination overhead. This paper proposes Scaling Plane, a two-dimensional scaling model that jointly optimizes the number of nodes and per-node multi-resource allocations (CPU, memory, network, IOPS). We introduce the novel concept of *diagonal scaling paths*—simultaneously adjusting both dimensions to approach the global optimum. Leveraging a smooth, approximate model of performance–cost–overhead trade-offs, we design DIAGONALSCALE, a discrete local search algorithm that computes Pareto-optimal configurations under SLA constraints. Experiments demonstrate that, compared to conventional single-dimensional scaling, our approach reduces p95 latency by up to 40%, lowers per-query cost by 37%, and cuts data rebalancing operations by 2–5×.

To address straggler effects and stale gradients caused by computational imbalance in distributed deep learning across heterogeneous computing environments (edge/cloud/HPC), this paper proposes the first adaptive batch-size scheduling framework grounded in proportional control theory. The framework integrates runtime worker load sensing, feedback-driven batch-size adjustment, and asynchronous SGD optimization to achieve fine-grained load balancing under dynamic resource conditions. Compared to baseline methods, it reduces training time by 14%–85% and improves model accuracy by up to 6.9% in asynchronous training settings. Its core innovation lies in introducing classical control theory into distributed training scheduling—enabling the first closed-loop, self-adaptive regulation of batch size—that jointly optimizes training efficiency and convergence quality.

Parsley, a group-based distributed hash table (DHT), suffers from limited stability and scalability in dynamic environments due to the absence of theoretical foundations for its group size parameters—specifically, the hard and soft limits. This work presents the first systematic modeling and analysis of how group size impacts overlay network performance. We propose a dual-bound mechanism comprising a soft target interval and a hard constraint, integrated with preemptive node relocation, dynamic data sharding, and formalized topology operation modeling. Extensive large-scale simulations validate the efficacy of our approach. Compared to state-of-the-art methods, our design significantly reduces group split/merge frequency under high churn, while improving load balancing and robustness. The framework provides both a theoretically grounded parameter configuration methodology and empirically validated guidelines—directly applicable to Parsley and other group-based DHTs.

This work addresses the performance bottleneck in distributed hash joins over wide-area networks caused by data skew, which leads to severe imbalance in computation and communication loads. To tackle this challenge, the authors propose Bala-Join, a novel approach that dynamically balances join workloads across geographically distributed SQL databases through an adaptive redistribution strategy. The core contributions include the Balanced Partitioning with Partial Replication (BPPR) algorithm, a distributed online skew-key detector, and the ASAP synchronization mechanism that integrates multicast-based redistribution, proactive signaling, and asynchronous pull. Experimental evaluation on real-world WAN deployments demonstrates that Bala-Join improves throughput by 25%–61% compared to state-of-the-art baselines while significantly reducing communication overhead and tail latency.

To address the scalability bottleneck imposed by cross-shard transaction scheduling in blockchain sharding systems—particularly under IoT, edge, and mobile workloads—this paper investigates efficient batched transaction scheduling. We propose two theoretically grounded and practically deployable scheduling frameworks: (i) a centralized scheduler based on bucket partitioning, which eliminates reliance on global state; and (ii) a distributed scheduler leveraging hierarchical clustering, supporting heterogeneous shard topologies and dynamic access patterns. To our knowledge, this is the first work to design an approximation algorithm for sharded blockchains with a provable competitive ratio upper bound of $O(A_{CS} cdot log d cdot log s)$. Experimental evaluation demonstrates that, compared to lock-based baselines, our approach reduces end-to-end latency by up to 3× and improves throughput by 2×, while significantly enhancing approximation quality under both chain-structured and random access patterns.

This work addresses the lack of a unified and comparable benchmark for fairly evaluating rule-based, learning-based, and large language model (LLM)-driven autoscaling strategies in big data batch processing scenarios. To this end, we propose BatchBench, an open-source, workload-aware benchmarking framework. BatchBench introduces a taxonomy encompassing six representative batch workload types, features a parameterized workload generator whose fidelity is validated via two-sample Kolmogorov–Smirnov tests and Earth Mover’s Distance, and defines a five-dimensional evaluation protocol covering cost, SLA compliance, responsiveness, scaling jitter, and interpretability. Notably, it enables, for the first time, side-by-side comparison of all three autoscaling strategy categories while incorporating LLM inference cost accounting. The framework’s design is complete, and its reference implementation will be open-sourced to establish a standardized experimental foundation for autoscaling research.

Automatic partitioning of large models across distributed accelerators faces three key challenges: an exponentially growing search space, high risk of out-of-memory (OOM) failures, and suboptimal or infeasible solutions due to heuristic pruning in existing tools. This paper proposes a novel hybrid approach integrating principled static compilation analysis with Monte Carlo Tree Search (MCTS). First, it models tensor dimension dependencies to precisely identify homogeneous sharding requirements and conflict constraints, thereby constructing a compact and feasible decision space. Second, it employs MCTS to efficiently explore this space while enforcing memory safety and execution efficiency. Evaluated across diverse hardware platforms and model architectures, our fully automated method discovers partitioning schemes that outperform industrial-grade baselines—including TensorFlow/XLA and DeepSpeed—achieving higher scalability, throughput, and zero GPU memory overflow.

To address the low efficiency and poor scalability of data quality assessment on large-scale datasets, this paper proposes the Chunked Data Shapley framework—a novel approach that integrates Shapley values with data chunking, optimized subset sampling, and single-iteration stochastic gradient descent to enable high-accuracy, high-efficiency quality evaluation. By partitioning data into chunks and estimating their marginal contributions, the framework rapidly identifies high-value or low-quality regions, supporting both classification and regression tasks. Experiments across multiple real-world tabular datasets demonstrate that our method achieves 80×–2300× speedup over state-of-the-art baselines while significantly improving the accuracy of low-quality sample identification. The framework thus establishes a scalable, theoretically grounded, and interpretable paradigm for data quality assessment in large-scale machine learning data governance.

This work addresses the practical challenges faced by d-choice load balancing in large-scale service systems, where bursty traffic, multi-priority tasks, and information noise significantly degrade load distribution efficiency and system stability. Bridging the gap between theoretical models and real-world deployment, this study systematically extends d-choice balancing along three critical dimensions: burst recovery, support for multiple task priorities, and tolerance to noisy state information. Leveraging large-scale simulations and an analytical framework based on generative models, the authors characterize and validate the policy’s behavior in dynamic, heterogeneous environments. The results demonstrate that the proposed strategy rapidly recovers from traffic bursts, effectively manages tasks of varying priorities, and remains robust under imperfect information, thereby offering a highly resilient scheduling solution for cloud-scale systems.

In multi-source, heterogeneous data sharing scenarios, efficiently selecting high-value datasets to enhance downstream model performance remains challenging. Method: This paper formally defines the “dataset-level selection” task and proposes a two-tier utility modeling framework that jointly captures heterogeneity across both datasets and data sources (e.g., institutions, domains), enabling few-shot generalization and adaptive selection under resource constraints. We introduce Dataset Selection via Hierarchies (DaSH), integrating hierarchical Bayesian modeling with utility propagation to jointly optimize active exploration and decision-making. Results: Experiments on Digit-Five and DomainNet demonstrate up to a 26.2% accuracy improvement over baselines, significantly reduced exploration steps, and strong robustness under low-resource conditions and critical data absence—establishing a new paradigm for principled, scalable dataset selection in heterogeneous federated settings.