Search papers, labs, and topics across Lattice.
This paper introduces ALPHABET, a compact linear-time sequence model that effectively compresses temporal history using stable complex pole modes, allowing for an explicitly auditable prediction interface. By leveraging a direct bank for modal state synthesis and an independent cascaded bank for analysis, ALPHABET retains critical temporal information while achieving competitive performance with only 6,437 parameters. The model demonstrates significant efficiency gains, achieving faster inference and training compared to nine baseline models, while maintaining high accuracy across a diverse set of tasks.
ALPHABET achieves Bayes oracle performance with a mere 6,437 parameters, revolutionizing efficiency in sequence modeling.
Can a sequence model remain competitive with only a few thousand parameters and an explicitly auditable prediction interface? We introduce ALPHABET, a compact linear-time model that compresses temporal history into stable complex pole modes: a direct bank synthesizes its modal states back into the feature trajectory, an independent cascaded bank analyzes the transformed trajectory without resynthesis, and an affine head reads only modal energies and lag moments from both banks. We characterize the temporal information this descriptor retains: for a stationary, fully observed feature process, each mode energy is a frequency-localized measurement of the second-order spectrum, the continuum of such measurements identifies the spectrum, and almost every mode separates any fixed finite set of spectrally distinct classes. On a Gaussian control with matched low-lag statistics, the learned descriptor approaches the Bayes oracle where raw autocovariances remain at chance. Across the fixed 82-task registry, ALPHABET attains mean rank 3.97 in the complete ten-family comparison. At the common-width D=64 runtime anchor, its 6,437 parameters deliver 5.02 times faster inference and 3.93 times faster complete training steps than the nine baselines on average.
