Search papers, labs, and topics across Lattice.
This paper introduces Prime Fourier Embeddings (PFE), a novel method for encoding integers using prime-indexed (cos, sin) pairs that leverage harmonic analysis to facilitate modular arithmetic. By structuring the representation to allow for direct selection of relevant prime channels, the authors demonstrate that linear maps on PFE exhibit a block-diagonal structure, confirming the theoretical predictions through empirical validation. The results reveal specialization ratios exceeding 500x between task-relevant and task-irrelevant channels, achieving perfect accuracy across various square-free composite moduli.
Prime Fourier Embeddings reveal that modular arithmetic can be efficiently structured, leading to over 500x specialization between relevant and irrelevant channels in neural networks.
Numbers have algebraic structure that standard neural embeddings often fail to expose. We introduce Prime Fourier Embeddings (PFE), which encode integers as prime-indexed (cos, sin) pairs derived from the harmonic analysis of Q, providing a pre-structured representation in which modular arithmetic reduces to selecting the relevant prime channel rather than discovering algebraic structure from scratch. We prove that any linear map equivariant with respect to the product group action on PFE must be block-diagonal with one independent block per prime -- a consequence of Schur's lemma applied to the resulting character decomposition. For square-free composite moduli, the Chinese Remainder Theorem predicts which prime channels are task-relevant. Both predictions are confirmed empirically: ablation studies show specialization ratios exceeding 500x between task-relevant and task-irrelevant channels, with perfect in-distribution test accuracy across all square-free composite moduli tested.
