Search papers, labs, and topics across Lattice.
This paper introduces BOOSTEDSOSA, a dual-FPGA machine learning-assisted scheduling architecture designed to enhance stochastic online scheduling in high-performance computing environments. By integrating a machine learning predictor that estimates job runtimes based solely on scheduler parameters at submission time, BOOSTEDSOSA significantly reduces the mean absolute error (MAE) in runtime estimates by up to 63.85% compared to traditional user estimates. The architecture achieves an impressive 17x speedup over an AVX-optimized software baseline, demonstrating its effectiveness in processing large volumes of jobs efficiently.
A machine learning predictor in BOOSTEDSOSA slashes runtime estimation errors by up to 63.85%, revolutionizing scheduling efficiency in high-performance computing.
Heterogeneous scheduling in stochastic, online envi- ronments, such as high-performance computing (HPC) systems, presents a significant challenge. Stochastic Online Scheduling Accelerators (SOSAs) offer a promising solution, but their effectiveness is compromised by a reliance on runtime estimates provided by users. These estimates introduce substantial vari- ance into the scheduling process (mean MAE in hundreds of Core-Days), thereby weakening the competitiveness of Stochastic Online Scheduling algorithms as their competitive-ratio bound increases with runtime variability. To address this limitation, we introduce BOOSTEDSOSA, a dual-FPGA ML-assisted Scheduling architecture that integrates a Machine Learning predictor for expected processing times, with a novel temporal-aware training policy. The predictor estimates job runtimes using only scheduler parameters available at submission time, enabling its use in existing HPC systems. Using historical real-world HPC job data (from the Argonne Leadership Comput- ing Facility, MIT Supercloud and UIUC Blue Waters workload datasets), we show that the predictor reduces MAE by up to 63.85% compared to user runtime estimates, and the additive training policy reduces MAE by up to 71.88% compared to a static model. End-to-end, BOOSTEDSOSA achieves an average 17x speedup over an AVX-optimized software baseline and processes up to 1,711 jobs/seconds
