Experimentation for Different Scheduling Policies on Queues: Mixed Differences-in-Q Estimators Based on Little's Law

2026-05-28

AI Summary

This work addresses the high bias and variance commonly observed in A/B tests of datacenter scheduling policies, which arise due to Markovian interference. To tackle this challenge, the paper introduces a novel hybrid causal inference method that uniquely integrates Little’s Law with a Differences-in-Q estimator. This approach explicitly models key complexities inherent in real-world queueing systems, including non-stationary arrival rates, heterogeneous service rates, and communication delays. Theoretical analysis and extensive simulations demonstrate that the proposed method substantially reduces both estimation bias and variance, achieving superior robustness and higher accuracy across a variety of practical scenarios compared to existing approaches.

Abstract

In data centers, tasks are dispatched to various servers to evenly distribute the workload. When a data center considers implementing a new scheduling algorithm, it typically conducts an A/B test prior to deployment to assess the real-world impact of this new method. However, a straightforward A/B test might be interfered with so-called ``Markovian'' interference. We utilized the Differences-in-Q estimator, as developed by Farias et al. (2022), and introduced mixed Differences-in-Q estimators grounded in Little's Law. We show that our A/B testing methods significantly reduce bias and variance when testing various scheduling policies. Extensive simulations were conducted under scenarios like non-stationary arrival rates, heterogeneous service rates, and communication delays. These simulations highlight the robustness and efficacy of our A/B testing approach.

Problem

• A/B testing
• scheduling policies
• Markovian interference
• queueing systems
• Little's Law

Innovation

• Differences-in-Q estimator
• Little's Law
• scheduling policy
• A/B testing
• Markovian interference