sorting algorithm design

This work addresses the scalability bottleneck of list ranking in large-scale distributed systems involving billions of elements and tens of thousands of processors. Building upon Sibeyn’s sparse dominating set algorithm, the authors achieve the first highly scalable solution for this classic problem at extreme scales through a co-design of algorithmic and performance optimizations, including indirect communication, exploitation of input locality, and message aggregation. The study provides a systematic quantitative analysis of how key parameters influence performance and demonstrates the approach’s high scalability and practicality across diverse structured inputs on up to 24,576 processor cores.

Current test-time computation scaling for LLMs in mathematical reasoning employs uniform resource allocation across subproblems, leading to insufficient resources for hard subproblems, redundant computation on easy ones, and diminishing marginal returns. Method: We propose a dynamic, difficulty-aware resource allocation framework: (1) decompose problems into subtasks; (2) estimate each subtask’s difficulty; and (3) adaptively select between fast, intuitive (System 1) or slow, deliberative (System 2) processing modes based on difficulty, while maintaining reasoning coherence via contextual propagation. This enables fine-grained, sequential computational scheduling. Results: On AIME25, our method improves accuracy by 13.75 percentage points and reduces computational cost by 33%–53% over uniform scaling baselines. It is the first work to introduce cognitively inspired, dynamic resource allocation for test-time LLM reasoning optimization.

This paper addresses the challenge of achieving both efficiency and low communication overhead in distributed sorting under both asynchronous and synchronous settings. We propose a minimalist randomized sorting algorithm based on harmonic distribution sampling, which repeatedly selects random index pairs and performs local sorting—naturally enabling asynchronous parallelism while smoothly transitioning to a low-communication synchronous variant. To our knowledge, this is the first harmonic sorter with an expected time complexity of $O(n(log n)^2)$, offering strong robustness, asynchronous scalability, and communication efficiency. We provide rigorous theoretical analysis proving near-linear speedup in both asynchronous and synchronous distributed environments, significantly outperforming classical comparison-based distributed sorting algorithms in terms of scalability and communication cost.

Sample inefficiency constitutes a fundamental bottleneck in large-scale parallel ranking and selection (R&S). Method: This paper proposes a novel “Cluster-and-Conquer” paradigm that inserts a lightweight, correlation-driven clustering step prior to classical divide-and-conquer—avoiding both high-precision correlation estimation and stringent clustering assumptions. Contribution/Results: We establish theoretical optimality in sample complexity; design the first robust parallel clustering algorithm tailored for R&S; and integrate gradient analysis with correlation modeling to enable seamless embedding into existing R&S pipelines. Experiments on real-world AI tasks—including neural architecture search—demonstrate substantial reductions in sampling overhead, achieving simultaneous theoretical optimality and practical performance gains.

This paper studies the scheduling of multi-class parallelizable jobs under limited server resources to minimize average response time. Jobs are categorized by parallelizability and size distribution, and must be dynamically assigned to $k$ servers. The work first reveals that the load regime fundamentally determines the optimal policy: Least-Parallelizable-First is asymptotically optimal in the sub-Halfin–Whitt (light-load) regime, while SERPT is asymptotically optimal in the super-NDS (heavy-load) regime. Leveraging this insight, we design an adaptive policy that requires no prior knowledge of system load. Using tools from multi-class queueing theory, load-scaling analysis, and online algorithm design, we rigorously prove that each regime-specific policy achieves the fundamental lower bound on average response time. Simulation results demonstrate that the proposed adaptive strategy consistently attains near-theoretically-optimal performance across the entire load spectrum.

This work addresses the challenges of workflow task composition in high-throughput, petabyte-scale data processing environments, where resource heterogeneity and execution overhead significantly impact performance. The authors propose a hybrid task composition strategy that dynamically balances task independence against execution grouping, formulated within a multi-objective optimization framework to achieve Pareto-optimal trade-offs among throughput, I/O cost, and CPU efficiency. Leveraging workflow DAG modeling and high-dimensional parameter space simulation, the approach enables policy-driven automated synthesis of workflows. Experimental results demonstrate that the proposed strategy achieves up to a 3.8× improvement in throughput and reduces network overhead by as much as 14.9× compared to baseline methods, offering a scalable workflow synthesis framework for extreme-scale scientific computing.

This study addresses the high computational complexity of traditional Divisible Load Theory (DLT) in determining optimal processing times for single-level tree networks, which hinders real-time scheduling and large-scale resource allocation. To overcome this limitation, the work introduces machine learning into DLT optimization for the first time, proposing a feedforward neural network–based prediction framework. The model implicitly learns load conservation and synchronized completion constraints from 16 engineered features derived from 100,000 synthetic data instances, bypassing explicit solution of DLT equations. Evaluated across diverse system configurations, the approach achieves R² accuracies of 97–99% (with MAPE of 1–5%) and sub-millisecond inference latency. This enables speedups of 10–100× over conventional methods while preserving near-optimal accuracy, thereby significantly facilitating real-time scheduling and design space exploration.

Sparse tensor algebra poses significant challenges for efficient load balancing on parallel systems due to its irregular structure, strong data dependencies, and skewed distributions. This work proposes the first provably load-balanced partitioning algorithm capable of handling arbitrary numbers of operands and multi-dimensional hierarchical sparsity patterns. The algorithm is integrated into a sparse tensor algebra compilation framework that automatically generates high-performance parallel kernels for both multicore CPUs and GPUs. By overcoming the limitations of conventional parallel merge strategies, the approach produces code that matches or exceeds the performance of hand-optimized libraries such as Intel MKL and cuSPARSE, achieving geometric mean speedups of 0.73–3.4× across diverse hardware platforms. Moreover, it substantially outperforms existing general-purpose parallelization strategies, with geometric mean speedups ranging from 2.0× to 6.4×.

Current large language model (LLM) inference services rely on generic heuristic strategies that overlook the unique dynamic structure of LLMs in request routing, scheduling, and KV cache management, leading to unstable performance and a lack of theoretical guarantees. This work presents the first systematic integration of operations research and machine learning systems to formally model the distinctive characteristics of LLM inference. Building upon this foundation, we propose an algorithmic framework grounded in mathematical optimization, queueing theory, and cache policy modeling. Our approach matches or surpasses existing heuristics across diverse workloads while providing provable performance bounds and enhanced predictability, thereby establishing a theoretically principled paradigm for LLM serving.

Traditional weak scalability analysis in visualization algorithm evaluation often suffers from inconsistent workload growth due to variations in data complexity or output size, making it difficult to accurately reflect an algorithm’s true scalability. This work systematically evaluates the impact of various data scaling strategies on the weak scalability of diverse visualization algorithms, revealing significant differences in their applicability across algorithms and datasets. To address this issue, the study proposes an improved methodology tailored for shared-memory environments that effectively mitigates inconsistencies in workload growth. Experimental results demonstrate that the proposed approach substantially enhances the consistency and accuracy of weak scalability assessments, offering a more reliable methodological foundation for evaluating the scalability of large-scale visualization algorithms.