Search papers, labs, and topics across Lattice.
This paper introduces SAGE, an innovative CLIP-based framework for vision-language time series forecasting that integrates temporal, cross-variable, textual, and visual information. By leveraging frequency-enhanced patches and variable-specific descriptions without relying on large language models during inference, SAGE achieves state-of-the-art accuracy across eight long-term benchmarks and the M4 competition. The results highlight the importance of multimodal alignment and variable-level knowledge in enhancing forecasting performance.
SAGE achieves state-of-the-art forecasting accuracy by integrating multimodal semantic knowledge without the computational burden of large language models.
Time series forecasting models operate on raw numerical sequences, lacking the semantic knowledge that domain experts implicitly leverage, such as the physical meaning of each variable, its statistical behavior, and its temporal dynamics. Recent efforts to bridge this gap fall into two camps. Some rely on large language models at inference time, which is computationally expensive. Others apply uniform textual prompts at the dataset level, ignoring the heterogeneous semantics across individual variates. We propose SAGE (Seeing and Augmenting with Grounded Encoding), an end-to-end CLIP-based framework that jointly models temporal, cross-variable, textual, and visual information. The CLIP text encoder processes frequency-enhanced patches and variable tokens, while gated residual paths inject variable-specific descriptions and statistical descriptors. In parallel, the frozen CLIP vision encoder aligns rendered series with temporal representations through a training-only contrastive objective. This dual use of CLIP adds complementary semantic and visual supervision without placing an LLM in the forecasting loop. Across eight long-term benchmarks and M4, SAGE achieves state-of-the-art accuracy. Ablations confirm complementary gains from multimodal alignment and variable-level knowledge.
