Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper benchmarks leading AI weather models (GenCast, FourCastNet 3, FGN) against ECMWF's IFS-ENS for tropical cyclone forecasting using novel probabilistic metrics. The study reveals that AI models, particularly FGN, significantly outperform the physics-based ensemble in cyclone tracking, achieving better probabilistic location scores for 90% of evaluated storms. While AI models exhibit limitations in intensity prediction and ensemble spread, their success in tracking suggests an emergent capability to capture predictive physical relationships beyond average weather conditions.
AI weather models now forecast tropical cyclone tracks with superior probabilistic accuracy to leading physics-based systems, even while struggling with intensity.
Weather prediction is undergoing an important shift, with data-driven AI models outperforming leading numerical weather prediction systems on average at a fraction of the computational cost. Yet, whether this success extends to extreme events remains contested. Earlier evaluations have exposed important shortcomings, but have largely focused on deterministic models and metrics, leaving the capabilities of the latest probabilistic models less understood. We conduct an extensive benchmark of tropical cyclone forecasting, comparing GenCast, FourCastNet 3, and FGN with ECMWF's operational physics-based ensemble, IFS-ENS. We develop a suite of probabilistic metrics and diagnostic analyses to assess cyclone tracking, intensity biases, uncertainty, and the wind-pressure relationship that characterizes storm structure. We find that, despite remaining limitations in intensity and ensemble spread, leading AI models substantially outperform IFS-ENS in cyclone tracking: FGN achieves better probabilistic location scores for 90% of evaluated storms in 2025. Further analyses reveal model-specific failure modes and suggest a lingering influence of pretraining data, pointing to opportunities for targeted improvement. Nonetheless, these results suggest that AI models are capable of capturing physical relationships that remain predictive well beyond typical weather conditions, motivating further investigation into how these capabilities emerge and how to strengthen them.