Search papers, labs, and topics across Lattice.
This paper introduces PINN-Cast, a continuous-depth transformer encoder for weather forecasting that leverages Neural ODEs within each encoder block to model smooth latent dynamics. It also incorporates a two-branch attention module with a derivative operator and a physics-informed training objective to enforce physical consistency. Experiments demonstrate PINN-Cast's improved performance in short-term weather forecasting compared to discrete transformers and existing Neural ODE forecasting methods.
Continuous-depth transformers, augmented with physics-informed loss, can significantly improve short-term weather forecasting, suggesting a promising path for hybrid physics-aware AI models.
Operational weather prediction has long relied on physics-based numerical weather prediction (NWP), whose accuracy comes at the cost of substantial compute and complex simulation workflows. Recent transformer-based forecasters offer efficient data-driven alternatives, however transformers are physics-agnostic models. Additionally, standard transformer encoders evolve representations through discrete layer updates that may be less suited to modeling smooth latent dynamics. In this work, we propose a continuous-depth transformer encoder for weather forecasting that integrates Neural Ordinary Differential Equation (Neural ODE) dynamics within each encoder block. Specifically, we replace discrete residual updates with ODE-based updates solved using adaptive numerical integration. We also introduce a two-branch attention module that combines conventional patch-wise self-attention with an auxiliary branch that applies a derivative operator to attention logits, providing an additional change-sensitive interaction signal. To further align forecasts with governing principles, we propose a customized physics-informed training objective that enforces physical consistency as a soft constraint. We evaluate the proposed method against a standard discrete transformer baseline and an existing continuous-time Neural ODE forecasting variant, demonstrating the importance of PINN-Cast in short term weather forecasting.
