Search papers, labs, and topics across Lattice.
This paper introduces Q-Interference, a novel quantum-inspired attention mechanism that enhances autoregressive language modeling by incorporating phase-aware interactions between token features. By augmenting each query and key with amplitude and learned phase, Q-Interference allows for constructive and destructive interference of token features, leading to richer attention dynamics. The authors address the memory challenges of naive implementations through an exact trigonometric factorization, enabling efficient computation while maintaining compatibility with existing Transformer architectures.
Q-Interference reveals that phase-aware attention can significantly enhance token interactions without the memory overhead of traditional methods.
GPT attention measures token compatibility through dot-product similarity. This mechanism is simple, effective, and memory-efficient. But it does not explicitly model whether strong token features should reinforce or suppress one another. We introduce Q-Interference, a fully classical quantum-inspired attention mechanism for autoregressive language modeling that augments each query and key feature with an amplitude and a learned phase. The resulting attention score is phase-aware which aligned phases contribute constructively while conflicting phases contribute destructively. Although Q-Interference yields a richer interaction rule than similarity alone, a naive implementation of Q-Interference requires a large token-pair-feature interaction tensor, making it memory-intensive and often impractical. To address this limitation, we propose an exact trigonometric factorization that computes the same score using two standard matrix multiplications avoiding materialization of the large intermediate tensor. Q-Interference fits directly into a Transformer block in GPT and leaves the remainder of the model architecture and next-token prediction objective unchanged. Experiments on public benchmark datasets and baseline models show that the proposed reformulation trains stably in a controlled GPT-style setting and provides a consistent memory advantage over naive phase-aware interference attention. These results support the specific contribution of this work: an exact memory-efficient reformulation that makes phase-aware interference attention practical within a standard GPT pipeline.
