Search papers, labs, and topics across Lattice.
This paper introduces TopoCast, a novel framework for evaluating the structural fidelity of Transformer-based time series forecasting models, addressing the limitations of traditional pointwise error metrics like Mean Squared Error (MSE). By utilizing Takens delay embedding and persistent homology, TopoCast reconstructs phase-space representations to derive a Topological Fidelity Score (TFS) and a Localized Topological Fidelity Score (LTFS), which account for temporal localization errors and oscillatory behavior. Experiments reveal that models with similar MSE can have significantly different structural fidelity, underscoring the importance of topological assessment in identifying overlooked failure modes in forecasts.
Models that seem equally accurate by traditional metrics can hide critical structural flaws, as TopoCast reveals through its innovative topological evaluation.
Deep learning-based models have achieved state-of-the-art performance in Time Series Forecasting (TSF), yet their evaluation remains dominated by pointwise error metrics such as Mean Squared Error (MSE), which quantify numerical accuracy but overlook structural properties of the forecast signal, including recurrent dynamics, oscillatory behavior, and phase alignment. As a result, forecasts exhibiting over-smoothing, phase shifts, or frequency distortions may achieve favorable error scores despite substantial structural degradation. To address this limitation, we propose TopoCast, a topology-driven framework for evaluating structural fidelity in TSF. TopoCast reconstructs phase-space representations of forecast and ground-truth sequences using Takens delay embedding and applies persistent homology to characterize their intrinsic dynamics. We derive four complementary topological fidelity measures from persistence diagrams and aggregate them into a Topological Fidelity Score (TFS). We further introduce dominant cycle overlap, a novel metric that maps persistent topological features to the temporal domain to assess whether dominant oscillatory patterns occur at the correct time points. Combined with TFS, this yields the Localized Topological Fidelity Score (LTFS), a phase-aware measure that captures temporal localization errors invisible to existing evaluation metrics. Experiments on five Transformer architectures across three real-world benchmark datasets demonstrate that models with similar forecasting errors can exhibit markedly different structural fidelity profiles, revealing failure modes overlooked by conventional evaluation and highlighting the value of topology-aware forecast assessment.
