Search papers, labs, and topics across Lattice.
This paper introduces a novel empirical evaluation methodology for predicting HPC job runtimes that emphasizes tail behavior, addressing the inadequacies of traditional averaging methods that obscure the impact of resource-intensive jobs. By leveraging GeoAccuracy weighted by resource usage and conducting decile analyses on production traces from NREL Eagle and ALCF Mira/Intrepid, the study reveals that UserReq outperforms XGBoost and Last2 in the top decile, achieving the highest GeoAccuracy and lowest underestimation rates. Implementing a hybrid scheduling policy based on these insights resulted in a significant reduction in mean wait times and a substantial increase in backfilled jobs, underscoring the importance of tail-aware evaluation in HPC scheduling.
Tail-focused evaluation reveals that traditional methods can misrepresent job runtime predictions, with UserReq outperforming other models in critical scenarios.
Runtime estimates affect reservation quality, backfilling opportunities, and queue delay in HPC schedulers. Under heavy tailed workloads, however, averaging over jobs can misrepresent scheduling impact because a small fraction of jobs dominates resource usage. This paper presents an empirical evaluation methodology for HPC job runtime prediction that focuses on the tail, combining GeoAccuracy weighted by resource usage with decile and split analyses. Using production traces from NREL Eagle and ALCF Mira/Intrepid, we compare XGBoost and Last2 against the user provided walltime estimate at submission (UserReq). Across all three datasets, evaluation focused on the tail changes the offline conclusion: MeanAccuracy keeps the methods relatively close, whereas GeoAccuracy reveals clearer separation and makes UserReq's strength in the upper tail visible. In the top decile, UserReq achieves the highest GeoAccuracy and lowest underestimation rate on all three datasets, and this pattern remains stable across rolling splits. We then translate this signal into a simple hybrid scheduling policy that keeps XGBoost for most jobs and routes the top decile by proxy_cost at submission to UserReq. Online replay on four production queues reduces mean wait time by up to 8% and increases backfilled jobs by 50% to 115%. These results show that offline evaluation focused on the tail better characterizes prediction quality relevant to scheduling and informs scheduling policy design.
