Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a practical methodology for MPI malleability that integrates seamlessly with existing high-performance computing (HPC) software and operational practices, addressing the limitations of fixed-size resource allocations in production HPC clusters. By implementing the Dynamic Management of Resources (DMR) framework, the authors demonstrate that their non-invasive approach can achieve performance on par with static baselines while significantly reducing node-hour consumption in two large-scale scientific applications across three TOP500 supercomputers. The findings highlight the feasibility of adopting dynamic resource management in production environments, making it easier for existing applications to benefit from enhanced resource flexibility.
Malleability in HPC can cut node-hour consumption significantly without sacrificing performance, paving the way for more efficient resource management in production environments.
Many large-scale scientific applications exhibit time-varying behavior, yet production HPC clusters still rely on rigid, fixed-size allocations, and most dynamic techniques remain confined to laboratory prototypes. This work presents a practical MPI malleability methodology that integrates with state-of-the-art high-performance computing (HPC) software stacks and operational practices. The methodology is implemented in the Dynamic Management of Resources (DMR) framework and is designed to ease adoption by existing applications without requiring intrusive code changes or scheduler modifications. We evaluate our approach by integrating the DMR API into two large-scale scientific applications and deploying them on three TOP500 supercomputers under realistic production configurations. Our non-invasive malleability solution achieves performance comparable to static baselines in controlled environments while substantially reducing node-hour consumption for identical workloads. These results show that malleability can be effectively exploited on production systems using vanilla resource managers, lowering the barrier to adoption of dynamic resource management in HPC.