Search papers, labs, and topics across Lattice.
This paper introduces a decision-focused reinforcement learning (DF-RL) framework that integrates a forecaster trained end-to-end with the charging policy of electric vehicles (EVs) to address the challenge of unknown departure times. By jointly training the forecaster and the RL agent, the authors mitigate the propagation of forecasting errors that can degrade decision quality in EV charging. The results show that DF-RL achieves up to a 14% improvement in total reward and a 55% reduction in unsupplied energy compared to traditional RL methods without departure time forecasting.
Integrating forecasting directly into the reinforcement learning loop can enhance EV charging decisions by significantly reducing energy waste and improving overall system performance.
The recent growth of EV adoption poses challenges for power systems, including increased peak demand and potential grid instability. Smart control of EV charging -- e.g., based on reinforcement learning (RL) -- can alleviate these issues by learning temporal and contextual patterns from historical data. Yet, in real-world scenarios, key features, such as departure time, often are unavailable. This, in turn, makes it harder for an RL agent to learn and execute an effective charging policy. To mitigate this uncertainty, a trained forecaster can approximate the unknown features from available data. However, since these forecasting models are typically trained for accuracy (rather than their impact on a downstream agent's decision quality), their errors may propagate and hinder the overall performance of a controller that is using the forecasts. To avoid this, we propose a decision-focused RL (DF-RL) framework in which the forecaster is trained end-to-end, i.e., with feedback from the charging policy actions taken by the RL agent. Such joint training of both the forecaster and controller ultimately results in higher-quality actions: our proposed DF-RL method yields superior charging decisions compared to other baselines, achieving up to a 14% improvement in total reward and a 55% reduction of unsupplied energy (i.e., charging that failed to happen because the EV already left), relative to the RL method without departure time forecasting.
