Search papers, labs, and topics across Lattice.
This paper introduces TempoWave, a novel temporal wavelet digit interface that enhances the integration of continuous numerical values into large language models (LLMs) for time series forecasting. By mapping scalar observations into digit-wise embeddings using multi-wavelet, multi-scale coefficients, TempoWave preserves numerical integrity and improves the model's ability to capture both local fluctuations and global trends. Experiments across five benchmarks reveal that TempoWave significantly outperforms standard numeric tokenization methods, establishing a new state-of-the-art in LLM-based forecasting accuracy.
TempoWave reveals that rethinking numerical embeddings can unlock significant improvements in LLM forecasting performance.
Large language models (LLMs) are attractive for context-aware time series forecasting because they can integrate heterogeneous textual signals, yet their discrete, language-oriented tokenization and embedding interfaces are misaligned with continuous numerical values, often harming numerical ordering and forecasting reliability. We propose TempoWave, a plug-and-play temporal wavelet digit interface that maps each scalar observation into digit-wise embeddings constructed from multi-wavelet, multi-scale coefficients. By directly overriding standard token representations, TempoWave seamlessly exposes both fine-grained local fluctuations and macro global structures in a transformer-compatible form, ensuring that precise numerical formatting, distinct digit identity, and robustness to common normalization operations are maintained throughout the LLM pipeline. Experiments across five context-enriched forecasting benchmarks demonstrate that TempoWave consistently improves LLM-based forecasters over standard numeric tokenization and alternative embedding interfaces, achieving a new state-of-the-art. These results highlight the numeric interface as a key bottleneck and suggest that principled multi-resolution embeddings can better couple LLMs' contextual reasoning with precise forecasting. Our code is available at https://github.com/DC-research/TempoWAVE and our model can be accessed at https://huggingface.co/Melady/TempoWAVE.
DeepMind