Search papers, labs, and topics across Lattice.
The paper introduces the Phasor Transformer, a novel transformer block that represents sequence states on the unit-circle manifold to address the quadratic complexity of dot-product self-attention in long-context time-series. It achieves global O(N log N) token mixing using trainable phase-shifts and parameter-free Discrete Fourier Transform (DFT) coupling, eliminating the need for explicit attention maps. Experiments on autoregressive time-series prediction demonstrate that the resulting Large Phasor Model (LPM) learns stable global dynamics and achieves competitive forecasting performance with a compact parameter budget.
Ditch quadratic attention bottlenecks: this new transformer variant achieves competitive time-series forecasting with O(N log N) complexity by representing sequence states on a unit circle.
Transformer models have redefined sequence learning, yet dot-product self-attention introduces a quadratic token-mixing bottleneck for long-context time-series. We introduce the \textbf{Phasor Transformer} block, a phase-native alternative representing sequence states on the unit-circle manifold $S^1$. Each block combines lightweight trainable phase-shifts with parameter-free Discrete Fourier Transform (DFT) token coupling, achieving global $\mathcal{O}(N\log N)$ mixing without explicit attention maps. Stacking these blocks defines the \textbf{Large Phasor Model (LPM)}. We validate LPM on autoregressive time-series prediction over synthetic multi-frequency benchmarks. Operating with a highly compact parameter budget, LPM learns stable global dynamics and achieves competitive forecasting behavior compared to conventional self-attention baselines. Our results establish an explicit efficiency-performance frontier, demonstrating that large-model scaling for time-series can emerge from geometry-constrained phase computation with deterministic global coupling, offering a practical path toward scalable temporal modeling in oscillatory domains.
