Affiliation:
1. a European Centre for Medium-Range Weather Forecasts, Reading, United Kingdom
2. b Department of Geography and Environmental Science, University of Reading, Reading, United Kingdom
Abstract
Abstract
The relative operating characteristic (ROC) curve is a popular diagnostic tool in forecast verification, with the area under the ROC curve (AUC) used as a verification metric measuring the discrimination ability of a forecast. Along with calibration, discrimination is deemed as a fundamental probabilistic forecast attribute. In particular, in ensemble forecast verification, AUC provides a basis for the comparison of potential predictive skill of competing forecasts. While this approach is straightforward when dealing with forecasts of common events (e.g., probability of precipitation), the AUC interpretation can turn out to be oversimplistic or misleading when focusing on rare events (e.g., precipitation exceeding some warning criterion). How should we interpret AUC of ensemble forecasts when focusing on rare events? How can changes in the way probability forecasts are derived from the ensemble forecast affect AUC results? How can we detect a genuine improvement in terms of predictive skill? Based on verification experiments, a critical eye is cast on the AUC interpretation to answer these questions. As well as the traditional trapezoidal approximation and the well-known binormal fitting model, we discuss a new approach that embraces the concept of imprecise probabilities and relies on the subdivision of the lowest ensemble probability category.
Publisher
American Meteorological Society
Reference20 articles.
1. Estimation of the reliability of ensemble-based probabilistic forecasts;Atger, F.,2004
2. Enhancing COSMO-DE ensemble forecasts by inexpensive techniques;Ben Bouallègue, Z.,2013
3. Quantile forecast discrimination ability and value;Ben Bouallègue, Z.,2015
4. Monitoring trends in ensemble forecast performance focusing on surface variables and high-impact events;Ben Bouallègue, Z.,2019
5. Accounting for representativeness in the verification of ensemble precipitation forecasts;Ben Bouallègue, Z.,2020
Cited by
11 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献