Search papers, labs, and topics across Lattice.
This paper critically evaluates the performance of three influential reinforcement learning (RL) orchestration systems against real-world challenges such as delayed telemetry and workload shifts. Contrary to prevailing assumptions, the authors find that most predicted performance degradations do not occur, revealing that many prior results are artifacts of evaluation choices rather than true indicators of robustness. The study highlights a significant institutional issue in AI research, where publication incentives prioritize benchmark improvements over genuine deployment performance, suggesting a need for more rigorous standards in the field.
Most reinforcement learning orchestration systems fail to demonstrate the robustness expected under real-world conditions, challenging the validity of existing benchmarks.
Reinforcement learning for service orchestration has been the subject of sustained research for over a decade, yet it is not used in production at scale. The usual explanation is that learned controllers degrade under delayed and noisy telemetry, workload shifts, and uncontrolled tenants. We test whether existing evidence supports that explanation. We evaluate three highly influential RL-based orchestration systems spanning resource allocation, DAG scheduling, and autoscaling, using pre-registered predictions about comparative degradation under production-relevant perturbations and paired inference with family-wise error correction. Across the tests, most predicted performance reversals do not occur. Diagnostic analyses show that these outcomes often reflect comparator collapse, artefact limitations, or evaluation choices rather than evidence that learned controllers tolerate the perturbations. One apparent advantage under observation lag is roughly fortyfold compared to a Kubernetes HPA-equivalent controller. Another widely cited result cannot be reconstructed from its released artefact, and the strongest reproducible margin is far smaller than the published results. Conclusions also reverse under changes in perturbation magnitude and evaluation mode. Based on these results and broader patterns in the literature, we identify an institutional problem. Publication and review incentives favour benchmark gains against convenient comparators, even when those gains provide little evidence of deployment performance. We argue that the problem is not solely technical. Rather, it is institutional, so learned orchestration needs production-grade comparators, registered perturbation models, separate operational metrics, and publication criteria that reward reproducible operational evidence. Without these changes, the literature can grow without establishing whether learning improves orchestration.
