Search papers, labs, and topics across Lattice.
This paper presents an LLM-based predictive scheduling system aimed at optimizing data center operations to enhance sustainability. By predicting execution time and energy consumption from source code, the system not only improves operational efficiency but also reduces environmental impact, achieving a 32% reduction in energy consumption and a 30% decrease in waiting time during real-world testing. The approach's fast inference and ability to generalize across tasks position it as a practical solution for addressing the growing energy demands of AI workloads.
Achieving a 32% reduction in energy consumption while cutting waiting times by 30% could redefine operational efficiency in data centers powered by LLMs.
The growing demand for AI-driven workloads, particularly from Large Language Models (LLMs), has raised concerns about the significant energy and resource consumption in data centers. This work introduces a novel LLM-based predictive scheduling system designed to enhance operational efficiency while reducing the environmental impact of data centers. Our system utilizes an LLM to predict key metrics such as execution time and energy consumption from source code, and it has the potential to extend to other sustainability-focused metrics like water usage for cooling and carbon emissions, provided the data center can track such data. The predictive model is followed by a real-time scheduling algorithm that allocates GPU resources, aiming to improve sustainability by optimizing both energy consumption and queuing delays. With fast inference times, the ability to generalize across diverse task types, and minimal data requirements for training, our approach offers a practical solution for data center scheduling. This framework demonstrates strong potential for advancing sustainability objectives in AI-driven infrastructure. Through our collaboration with a data center, we achieved a 32% reduction in energy consumption and a 30% decrease in waiting time.
