Search papers, labs, and topics across Lattice.
This paper addresses the scheduling challenges in Uncertainty Quantification (UQ) workflows within high-performance computing by introducing a dynamic load balancer in the UQ and Modelling Bridge (UM-Bridge) framework. The load balancer effectively manages heterogeneous model evaluations with loose dependencies, demonstrated through a Bayesian inverse problem involving multilevel delayed acceptance sampling and tsunami simulations. The results show that the load balancer significantly reduces average node idle time to nearly a millisecond, optimizing the distribution of sampling requests without prior workload assumptions.
A dynamic load balancer can reduce node idle time to nearly a millisecond in complex UQ workflows, revolutionizing how we approach scheduling in high-performance computing.
Uncertainty Quantification (UQ) workflows present a particular scheduling challenge in high performance computing environments, as they typically generate large numbers of heterogeneous model evaluations with loose but non-trivial dependencies between tasks. A static one-size-fits-all approach in traditional schedulers is inadequate to handle heterogeneous tasks optimally. We introduce an improved load balancer in the UQ and Modelling Bridge (UM-Bridge) framework aimed at mitigating these issues; UM-Bridge is a language-agnostic interface developed to couple UQ software with numerical simulation. As a realistic example, we test the load balancer with a Bayesian inverse problem solved via multilevel delayed acceptance sampling. The underlying forward problem is a hierarchy of tsunami simulations enabled through ExaHyPE, whose runtimes span several orders of magnitude and loose dependencies between levels make the workload particularly challenging to schedule. Our results indicate the load balancer is effective at distributing the sampling requests with an average node idle time of close to a millisecond, while not making any prior assumptions about the workload.
