Search papers, labs, and topics across Lattice.
This paper introduces an explainable deep reinforcement learning (XRL) framework tailored for optimal energy management in buildings, addressing the complexities introduced by renewable energy sources. By training both on-policy and off-policy DRL agents on a comprehensive state space that includes real-time data and external signals, the study reveals that on-policy methods like A2C and PPO significantly outperform off-policy approaches in terms of cumulative rewards and stability. The XRL framework not only optimizes battery management to reduce electricity costs but also enhances user trust by providing transparent insights into the decision-making processes of the agents.
On-policy deep reinforcement learning agents can significantly outperform off-policy methods in optimizing energy management while providing clear insights into their decision-making.
The increasing integration of renewable energy sources into power systems, particularly in buildings equipped with photovoltaic (PV) panels and energy storage systems, introduces significant complexity in energy systems. Volatile power generation, varying electricity tariffs, and increased entities, e.g., PV systems, and heat pumps, have increased the complexity and made the system harder to operate. This leads to the demand for additional control and optimization routes including data-based controls, such as reinforcement learning. While deep reinforcement learning (DRL) has emerged as a promising solution to optimize building operations in dynamic and ever more complex environments, its black-box nature impedes user trust and practical adoption. This paper presents a framework for explainable deep reinforcement learning (XRL) applied to energy management in residential buildings. We demonstrate its usage on both synthetic data but also on real-world data from the Living Lab Energy Campus (LLEC) at KIT. We train and compare both on-policy and off-policy DRL agents on an expanded state space that incorporates real-time measurements (demand, PV generation, battery power, state of charge), external signals (dynamic electricity price, local weather data), calendrical and holiday indicators, and forecasts for demand and price. Our experimental results indicate that on-policy algorithms, particularly Advantage Actor Critic (A2C) and Proximal Policy Optimization (PPO), outperform off-policy methods in terms of cumulative rewards and policy stability. To explain these models, we employ post-hoc interpretation techniques to elaborate the learned control policies. Our findings demonstrate that the XRL framework not only reduces electricity costs through optimal battery management, but also provides transparent, actionable insights into the agent's decision-making process.
