Search papers, labs, and topics across Lattice.
This paper introduces FAiT, a Frequency-Aware Inverted Transformer designed to enhance Multivariate Time Series Forecasting by addressing the limitations of traditional Transformer architectures that act as low-pass filters. By employing Inverted Attention to create a high-pass branch and utilizing Dynamic Temporal-Frequency Modulation (DTFM) for adaptive spectral weight calibration, FAiT effectively captures evolving temporal dynamics in time series data. Experimental results show that FAiT outperforms existing state-of-the-art models while ensuring computational efficiency, highlighting its potential for improved forecasting accuracy in dynamic environments.
FAiT outperforms traditional Transformers by capturing evolving spectral characteristics in time series data, enabling sharper forecasts of transient signals.
While Transformer-based architectures have established themselves as a dominant paradigm in Multivariate Time Series Forecasting (MTSF), their core self-attention mechanism inherently functions as a low-pass filter, systematically smoothing out high-frequency signals vital for sharp local changes. Recent advancements have increasingly incorporated frequency-domain operations to address this bias, however, most existing designs rely on fixed spectral bases and apply sequence-wise (uniform) modulation, implicitly assuming a time-invariant frequency response. This overlooks a key property of real-world series that their spectral characteristics often evolve over time, making uniform modulation insufficient for capturing fine-grained temporal dynamics. To tackle these limitations, we propose FAiT, a Frequency-Aware inverted Transformer. Specifically, FAiT rectifies the spectral bias internally through Inverted Attention, which interprets the attention map as a learnable low-pass operator and constructs a dedicated complementary high-pass branch by inverting the attention matrix to recover attenuated transient signals. Furthermore, FAiT introduces Dynamic Temporal-Frequency Modulation (DTFM), which synthesizes instance-conditioned weights to adaptively re-calibrate the energy of spectral sub-bands, enabling fine-grained control over evolving multi-scale patterns. Extensive experiments on widely used benchmarks demonstrate that FAiT consistently outperforms state-of-the-art Transformer-based and frequency-enhanced baselines, while maintaining computational efficiency.
