Anticipated Verification: A Simple Method for Representing Confidence and Trust in Metocean Forecast Information to Effectively Enhance Operational Decision-Making

Abstract

Abstract It is easy to look back at a past forecast and assess how well it performed relative to the observed weather, but it is an obvious fact that such verification is only possible after the event has passed. Despite advances in skill in predicting variables such as wind speed or significant wave height, a perennial problem for offshore industry users interested in the incisive application of metocean forecast information to effectively enhance their operational decision-making is knowing how much ‘trust’ to place in these data at lead times beyond a few days ahead. Here, for the first time, we propose a method for the stratification of (past) verification statistics that applies unused information from the (future) forecast to anticipate how well the numerical weather prediction will perform a priori, providing a ‘weather-aware’ estimate of its skill (confidence) before the event has occurred. Stratification is a method for being able to identify differences in characteristics of a particular subset of data, rather than just considering the whole population together. Since forecast performance is partly determined by the ‘drivers of predictability’ at that particular forecast horizon (e.g. prevailing atmosphere/ocean regime), we apply stratification on the basis of the classification of the large-scale weather present at the time to ‘filter’ past verification statistics, as considered in terms of the continuous ranked probability skill score. When used in real-time with a future forecast, the technique offers a refined estimate of the skill by taking advantage of the context of the weather being predicted. The translation of this ‘anticipated verification’ information into a qualitative description offers a robust indicator of confidence, suitable for (dynamically) guiding user trust in the forecast. Using 2.75 years of archived forecast data from the European Centre for Medium-Range Weather Forecasts (ECMWF) extended-range (monthly lead time) ensemble prediction system (EPS) for an example location in the North Sea, the derivation of a dynamic (conditionally sampled) ‘trust index’ is presented; the application of which being evaluated using trials on unseen data from the same site. The benefit of the approach demonstrates significant potential for improving confidence in decision-making from individual short- to medium- range forecasts, as well as building trust in longer-range forecasts. While not a substitute for meteorologist guidance regarding the comprehensiveness of the various drivers of predictability at sub-seasonal timescales, this is none-the-less an important step in making such forecasts usable, useful and used.