Quantification of NSSL Warn-on-Forecast System Accuracy by Storm Age Using Object-Based Verification

Abstract

Abstract The National Severe Storm Laboratory’s Warn-on-Forecast System (WoFS) is a convection-allowing ensemble with rapidly cycled data assimilation (DA) of various satellite and radar datasets designed for prediction at 0–6-h lead time of hazardous weather. With the focus on short lead times, WoFS predictive accuracy is strongly dependent on its ability to accurately initialize and depict the evolution of ongoing storms. Since it takes multiple DA cycles to fully “spin up” ongoing storms, predictive skill is likely a function of storm age at the time of model initialization, meaning that older storms that have been through several DA cycles will be forecast with greater accuracy than newer storms that initiate just before model initialization or at any point after. To quantify this relationship, we apply an object-based spatial tracking and verification approach to map differences in the probability of detection (POD), in space–time, of predicted storm objects from WoFS with respect to Multi-Radar Multi-Sensor (MRMS) reflectivity objects. Object-tracking/matching statistics are computed for all suitable and available WoFS cases from 2017 to 2021. Our results indicate sharply increasing POD with increasing storm age for lead times within 3 h. PODs were about 0.3 for storm objects that emerge 2–3 h after model initialization, while for storm objects that were at least an hour old at the time of model initialization by DA, PODs ranged from around 0.7 to 0.9 depending on the lead time. These results should aid in forecaster interpretation of WoFS, as well as guide WoFS developers on improving the model and DA system. Significance Statement The Warn-on-Forecast System (WoFS) is a collection of weather models designed to predict individual thunderstorms. Before the models can predict storms, they must ingest radar and satellite observations to put existing storms into the models. Because storms develop at different times, more observations will exist for some storms in the model domain than others, which results in WoFS forecasts with different accuracy for different storms. This paper estimates the differences in accuracy for storms that have existed for a long time and those that have not by tracking observed and predicted storms. We find that the likelihood of WoFS accurately predicting a thunderstorm nearly doubles if the storm has existed for over an hour prior to the forecast. Understanding this relationship between storm age and forecast accuracy will help forecasters better use WoFS predictions and guide future research to improve WoFS forecasts.