Search papers, labs, and topics across Lattice.
This paper introduces SMART-MIG, a parallel computing system that leverages Mean-Field Multi-Agent Reinforcement Learning (MF-MARL) to optimize GPU scheduling and partitioning for energy efficiency in Multi-Instance GPU (MIG) environments. The framework addresses the dual challenges of job scheduling and GPU partitioning, maintaining constant complexity in repartitioning as job and GPU counts increase. Experimental results demonstrate an 18% improvement in energy-tardiness efficiency over static partitioning, while remaining only 27% above the theoretical lower bound for energy consumption.
Achieving an 18% boost in energy-tardiness efficiency, SMART-MIG redefines GPU scheduling for energy-conscious machine learning applications.
The emergence of Multi-Instance GPU (MIG) technology enables us to run smaller machine learning models on partitions of a GPU rather than the entire device, thus improving utilization and reducing energy consumption, albeit with potential performance trade-offs. Meanwhile, the growing energy demands of GPU-equipped data centers motivate the development of online partitioning and scheduling schemes that not only ensure fast job processing but also achieve high energy efficiency. However, achieving energy-tardiness efficiency with manageable algorithmic complexity in large-scale scheduling remains a great challenge, due to the dual objectives of deciding on the GPU partitions and scheduling jobs onto the slices of the heterogeneous partitions. To address this challenge, we propose SMART-MIG, a parallel computing system that combines Mean-Field Multi-Agent Reinforcement Learning (MF-MARL) for large-scale MIG repartitioning with tailored heuristic algorithms for job scheduling. We demonstrate that the complexity of the repartitioning component remains constant even as the number of jobs and GPUs increases. We also establish theoretical lower bounds on energy consumption and tardiness to rigorously benchmark system performance. Finally, extensive experiments show that SMART-MIG improves the energy-tardiness efficiency by $18\%$ compared to its corresponding static-partitioning counterpart, while being only $27\%$ above the theoretical lower bound on energy consumption.
