Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Generative precipitation nowcasting models are predominantly trained and evaluated on wet-biased datasets, which over-represent precipitation events and skew evaluation towards matching this distribution. This target-dependent sampling, even with an ideal model, masks training biases by aligning learned distributions with evaluation biases. Consequently, a fixed diffusion model exhibits contradictory calibration assessments (under-predictive on wet-biased test sets, over-predictive on deployment-representative sets), misdirecting downstream development.
Wet-biased data pipelines can make even ideal generative precipitation models appear poorly calibrated, masking fundamental training biases and misdirecting research.
Generative models for precipitation nowcasting are trained and evaluated predominantly on wet-biased datasets, which over-represent precipitation events to emphasize rare extremes. Selecting test cases based on target observations is known to favor forecasts that match the selected distribution rather than the intended deployment distribution. We show analytically how this target-dependent sampling also shifts the conditional distribution learned by generative models. When training and test sampling match, even an ideal model learns precisely the distribution favored by evaluation, allowing the evaluation bias to conceal the training bias. Using a fixed satellite-to-radar diffusion model, we demonstrate that target-dependent test sampling can reverse the calibration assessment. The model appears under-predictive on a wet-biased test set but over-predictive on a deployment-representative test set. This result is replicated outside the model's training domain after a seven-year gap. Utilizing classifier-free guidance as an example, we demonstrate how the contradictory calibration assessment can misdirect downstream model development.