Weighted verification tools to evaluate univariate and multivariate forecasts for high-impact weather events

by   Sam Allen, et al.

To mitigate the impacts associated with adverse weather conditions, meteorological services issue weather warnings to the general public. These warnings rely heavily on forecasts issued by underlying prediction systems. When deciding which prediction system(s) to utilise to construct warnings, it is important to compare systems in their ability to forecast the occurrence and severity of extreme weather events. However, evaluating forecasts for extreme events is known to be a challenging task. This is exacerbated further by the fact that high-impact weather often manifests as a result of several confounding features, a realisation that has led to considerable research on so-called compound weather events. Both univariate and multivariate methods are therefore required to evaluate forecasts for high-impact weather. In this paper, we discuss weighted verification tools, which allow particular outcomes to be emphasised during forecast evaluation. We review and compare different approaches to construct weighted scoring rules, both in a univariate and multivariate setting, and we leverage existing results on weighted scores to introduce weighted probability integral transform (PIT) histograms, allowing forecast calibration to be assessed conditionally on particular outcomes having occurred. To illustrate the practical benefit afforded by these weighted verification tools, they are employed in a case study to evaluate forecasts for extreme heat events issued by the Swiss Federal Office of Meteorology and Climatology (MeteoSwiss).


Evaluating forecasts for high-impact events using transformed kernel scores

It is informative to evaluate a forecaster's ability to predict outcomes...

Extreme events evaluation using CRPS distributions

Verification of ensemble forecasts for extreme events remains a challeng...

Shootout-89: A Comparative Evaluation of Knowledge-based Systems that Forecast Severe Weather

During the summer of 1989, the Forecast Systems Laboratory of the Nation...

Assessing the calibration of multivariate probabilistic forecasts

Rank and PIT histograms are established tools to assess the calibration ...

Photographic Visualization of Weather Forecasts with Generative Adversarial Networks

Outdoor webcam images are an information-dense yet accessible visualizat...

Bad and good errors: value-weighted skill scores in deep ensemble learning

In this paper we propose a novel approach to realize forecast verificati...

Robustness Evaluation of Regression Tasks with Skewed Domain Preferences

In natural phenomena, data distributions often deviate from normality. O...

Please sign up or login with your details

Forgot password? Click here to reset