Engineering MultiQueues: Fast Relaxed Concurrent Priority Queues
Traditional concurrent priority queues often become performance bottlenecks under parallel workloads, limiting throughput and increasing latency. This paper proposes MultiQueues—a high-throughput, low-latency concurrent priority queue with relaxed semantics. Methodologically, it employs a multi-sharded queue architecture, triple batching (for insertions, deletions, and load balancing), and a novel wait-free locking technique to enable lock-free transformation from sequential heap structures into efficient concurrent relaxed ones. It further introduces two quantitative metrics—rank error and delay—to rigorously characterize relaxation quality. Experimental evaluation on representative workloads—including online scheduling and discrete-event simulation—demonstrates 3–10× higher throughput over state-of-the-art designs, while maintaining bounded rank error and stable, low latency. MultiQueues thus achieves superior scalability and predictability without sacrificing practicality.