Search papers, labs, and topics across Lattice.
This paper introduces PhysMetrics.Weather, a novel evaluation framework designed to assess the physical consistency of machine learning weather prediction models. By employing three distinct types of metrics鈥攃onservation, spectral, and dynamical鈥攖his framework addresses the critical gap in ensuring that MLWP forecasts align with established physical laws, rather than relying solely on traditional pixel-wide error metrics. The key finding is that incorporating physical realism metrics can significantly enhance the reliability and operational readiness of MLWP models, guiding the development of more robust, physics-informed architectures.
ML weather models may excel in accuracy, but without physical consistency, their forecasts could mislead operational decisions.
Machine learning weather prediction (MLWP) models have achieved impressive forecasting performance at a small fraction of the computational costs required for traditional physics-based methods. However, they are primarily (1) data-driven and (2) evaluated using pixel-wide error metrics (e.g., RMSE), so there are no guarantees that their forecasts are consistent with known physical laws. We introduce PhysMetrics.Weather, an evaluation framework that assesses the physical realism of MLWP models across three types of metrics: conservation, spectral, and dynamical. By quantifying physical realism, this tool guides the development of physics-informed architectures and helps evaluate whether MLWP models are reliable for operational use. Our framework is available on Github at https://github.com/Emmakast/PhysMetrics.Weather.
