Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study enhances reinforcement learning (RL) for residential heat pump control by incorporating a compressor-wear term into the reward function, addressing the issue of on-off cycling that leads to compressor wear. By training both Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO) on the same Markov decision process, the authors demonstrate that SAC successfully learns a continuous modulation policy, achieving zero start-ups per day and significantly reducing thermal discomfort. In contrast, PPO resorts to a less efficient bang-bang control strategy, resulting in increased cycling compared to the baseline.
SAC's innovative approach eliminates compressor cycling entirely, slashing thermal discomfort by up to 90.7% at a minimal cost increase.
On--off cycling is the main cause of compressor wear in residential heat pumps, yet reinforcement learning (RL) controllers for buildings typically optimise only energy cost and thermal comfort, ignoring how much the learned policy cycles. We add a levelised compressor-wear term to the control reward and study how the resulting behaviour depends on the RL algorithm. Training Soft Actor---Critic (SAC) and Proximal Policy Optimisation (PPO) on an identical Markov decision process for the BOPTEST bestest hydronic heat pump case, we find that SAC learns a continuous modulation policy that keeps the compressor permanently engaged---the operating principle of an inverter-driven heat pump---achieving zero start-ups per day, whereas PPO collapses to bang-bang control that cycles more than the baseline. On the BOPTEST emulator the SAC policy cuts thermal discomfort by up to 90.7% for an 11.5% cost increase, while eliminating all baseline cycling.