Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces a novel approach to optimize thermal energy storage (TES) scheduling in buildings by adapting an open-weight reasoning model through reinforcement learning with verifiable rewards (RLVR). By converting dynamic programming (DP) action values into dense rewards, the method effectively trains the model to output heat-pump setpoints with only 30 training prompts, achieving emissions reductions from 70.5 to 61.2 kg-CO2, closely approaching the DP optimum. The findings highlight the importance of inference-time reasoning, as GPT-5 nearly matches DP and model predictive control (MPC) performance without task-specific training, while a non-reasoning model underperforms significantly.
Reinforcement fine-tuning can cut thermal emissions in building energy management, bringing open-weight reasoning models closer to optimal performance with minimal training data.
Buildings are expected to shift cooling loads in response to grid conditions. Thermal energy storage (TES) enables this shift, but scheduling it well requires planning hours ahead under storage constraints. Model predictive control (MPC) and reinforcement learning are difficult to scale across buildings. This study instead adapts an open-weight reasoning model through reinforcement learning with verifiable rewards (RLVR). We convert exact offline dynamic-programming (DP) action values into dense rewards for every candidate action. Using only 30 training prompts, reinforcement fine-tuning (RFT) trains the model as an upper-level scheduler that outputs hourly heat-pump setpoints from text-based states and forecasts. Evaluation uses a deliberately simple office-building TES benchmark where exact DP is tractable and the optimum is known. RFT reduces the open-weight model's emissions from 70.5 to 61.2 kg-CO2, close to the DP optimum of 60.8 kg-CO2. GPT-5 nearly matches DP and MPC without task-specific training, while GPT-4o, a non-reasoning LLM, produces higher emissions than the no-storage baseline, so inference-time reasoning appears important. Trace analysis shows that RFT mainly stabilizes observable planning patterns (candidate comparison, look-ahead, and feasibility checking) rather than creating a new strategy. Robustness and generalization tests clarify what transfers: the reinforced planning patterns persist under forecast errors and an unseen TES condition and carry over to a battery task, but its different structure limits the gains. DP-based verifiable rewards offer a practical way to adapt open-weight reasoning models to building storage scheduling. These results motivate higher-fidelity tests of whole-building control and scalable verifiers for city-scale energy management.