Incentives and Evidence in Learned Service Orchestration

2026-06-15

AI Summary

Reinforcement learning (RL) has long remained absent from real-world service orchestration deployments, commonly attributed to telemetry latency, load fluctuations, and tenant-induced unpredictability—yet systematic empirical validation is lacking. This study rigorously evaluates performance degradation of three representative RL-based orchestration systems under production-grade perturbations, employing preregistered experiments, paired statistical inference, and family-wise error correction. Results reveal that most performance gains claimed in the literature either fail to replicate or diminish substantially; only one system consistently outperforms Kubernetes Horizontal Pod Autoscaler (HPA) by approximately 40× under observational delay. The findings indicate that existing advantages often stem from inadequate baselines, limited artifacts, or evaluation biases, underscoring an urgent need for deployment-oriented evaluation standards and institutional incentives to bridge the gap between research and practice.

Abstract

Reinforcement learning for service orchestration has been the subject of sustained research for over a decade, yet it is not used in production at scale. The usual explanation is that learned controllers degrade under delayed and noisy telemetry, workload shifts, and uncontrolled tenants. We test whether existing evidence supports that explanation. We evaluate three highly influential RL-based orchestration systems spanning resource allocation, DAG scheduling, and autoscaling, using pre-registered predictions about comparative degradation under production-relevant perturbations and paired inference with family-wise error correction. Across the tests, most predicted performance reversals do not occur. Diagnostic analyses show that these outcomes often reflect comparator collapse, artefact limitations, or evaluation choices rather than evidence that learned controllers tolerate the perturbations. One apparent advantage under observation lag is roughly fortyfold compared to a Kubernetes HPA-equivalent controller. Another widely cited result cannot be reconstructed from its released artefact, and the strongest reproducible margin is far smaller than the published results. Conclusions also reverse under changes in perturbation magnitude and evaluation mode. Based on these results and broader patterns in the literature, we identify an institutional problem. Publication and review incentives favour benchmark gains against convenient comparators, even when those gains provide little evidence of deployment performance. We argue that the problem is not solely technical. Rather, it is institutional, so learned orchestration needs production-grade comparators, registered perturbation models, separate operational metrics, and publication criteria that reward reproducible operational evidence. Without these changes, the literature can grow without establishing whether learning improves orchestration.

Problem

• service orchestration
• reinforcement learning
• production deployment
• evaluation bias
• institutional incentives

Innovation

• reinforcement learning
• service orchestration
• reproducibility
• production-grade evaluation
• institutional incentives