Search papers, labs, and topics across Lattice.
This paper introduces a two-stage, model-agnostic framework for time-series forecasting that separates the tasks of prediction and residual learning to address systematic biases in single-stage transformer architectures. By employing a base transformer for initial predictions followed by a dedicated meta-corrector to refine residuals, the method captures structured error patterns while preserving cross-variable dependencies. Evaluated across eight benchmark datasets, the approach achieves state-of-the-art performance, significantly reducing mean squared error and mean absolute error, thereby enhancing the robustness of transformer-based models to complex temporal dynamics.
Systematic biases in transformer-based time-series forecasting can be effectively mitigated by decoupling prediction and residual learning into a two-stage framework.
Transformer-based models have emerged as leading paradigms in time-series forecasting in recent years, employing self-attention mechanisms to capture long-range dependencies. Despite their success, these single-stage forecasting architectures exhibit persistent systematic residual biases arising from structural discrepancies, unmodeled stochastic components, or inadequate multi-scale temporal representations. This limitation persists when residuals are treated as irreducible noise, precluding adaptive correction of structured error patterns. To address this limitation, we introduce a two-stage, model-agnostic framework that explicitly decouples forecasting and residual learning into distinct stages of representation learning. A base transformer first generates the initial predictions. Subsequently, a dedicated meta-corrector dynamically models structured error patterns across multivariate channels, preserves cross-variable dependencies, and iteratively refines the residual bias of the base transformer. By formalizing this pipeline as a hypothesis space expansion, our framework addresses approximation limitations inherent in single-stage architectures, removes reliance on restrictive assumptions, and enables end-to-end learning of complex error dynamics. Evaluated on eight popular benchmark datasets using established protocols, our approach achieves state-of-the-art performance, with significant improvements in standard metrics (MSE, MAE). The results demonstrate the framework's ability to mitigate systematic biases and enhance robustness to complex temporal dynamics, advancing the practical applicability of transformer-based forecasting models.
