Search papers, labs, and topics across Lattice.
This paper compares five deep learning models, including a novel Deep Autoencoder-Transformer, for energy demand forecasting using multivariate time series data. The study addresses the need for accurate energy demand prediction to optimize consumption and maintain grid stability by benchmarking CNN-LSTM, Bidirectional LSTM, GRU, Transformer, and the proposed hybrid model. Results show that the Deep Autoencoder-Transformer achieves the best performance, with MAE = 8.5, RMSE = 10.75, MAPE = 3.46%, and R2 = 0.991, demonstrating the effectiveness of attention mechanisms and autoencoding for capturing complex temporal patterns.
Autoencoder-Transformer hybrids crush vanilla Transformers at energy demand forecasting, hinting at a powerful recipe for multivariate time series problems.
This study evaluates five prominent deep learning models—CNN-LSTM, Bidirectional LSTM, GRU, Transformer, and the proposed Deep Autoencoder-Transformer for the task of energy demand forecasting. Accurate prediction of energy demand is essential for optimizing consumption and maintaining power grid stability amidst increasing complexity and multivariate data characteristics. While previous research has predominantly assessed more traditional models such as LSTM and GRU, this research fills an important gap by thoroughly comparing these with the Transformer and a novel hybrid autoencoder-Transformer model. The models were systematically trained on multivariate inputs after comprehensive preprocessing and evaluated using statistical metrics including MAE, RMSE, MAPE, and coefficient of determination (R2). The findings demonstrate that the Deep Autoencoder-Transformer model outperforms all other architectures, achieving the lowest error rates (MAE = 8.5, RMSE = 10.75, MAPE = 3.46%) and highest explanatory power (R2 = 0.991). The Transformer also achieves strong performance (MAE = 10.14, R2 = 0.988), reflecting its ability to model long-term dependencies effectively. GRU and Bidirectional LSTM models follow, balancing accuracy and computational efficiency, while CNN-LSTM, despite its combined spatial and temporal feature extraction abilities, shows comparatively lower precision likely due to architectural limitations with long-range temporal modeling. This study highlights the superior capability of attention-based Transformer architectures, especially when combined with deep autoencoding, to capture complex temporal patterns in multivariate energy data. It offers a scalable and systematic framework for benchmarking deep learning models applicable to energy demand forecasting. These insights are valuable to energy system operators and policymakers for selecting appropriate machine learning models, with the hybrid Deep Autoencoder-Transformer emerging as a promising solution for more accurate, long-horizon, multi-step forecasting in intelligent energy systems.
