A Dynamical Performance-Ranking Method for Predicting Individual Ensemble Member Performance and Its Application to Ensemble Averaging

Abstract

Abstract This study proposes a dynamical performance-ranking method (called the Du–Zhou ranking method) to predict the relative performance of individual ensemble members by assuming the ensemble mean is a good estimation of the truth. The results show that the method 1) generally works well, especially for shorter ranges such as a 1-day forecast; 2) has less error in predicting the extreme (best and worst) performers than the intermediate performers; 3) works better when the variation in performance among ensemble members is large; 4) works better when the model bias is small; 5) works better in a multimodel than in a single-model ensemble environment; and 6) works best when using the magnitude difference between a member and its ensemble mean as the “distance” measure in ranking members. The ensemble mean and median generally perform similarly to each other. This method was applied to a weighted ensemble average to see if it can improve the ensemble mean forecast over a commonly used, simple equally weighted ensemble averaging method. The results indicate that the weighted ensemble mean forecast has a smaller systematic error. This superiority of the weighted over the simple mean is especially true for smaller-sized ensembles, such as 5 and 11 members, but it decreases with the increase in ensemble size and almost vanishes when the ensemble size increases to 21 members. There is, however, little impact on the random error and the spatial patterns of ensemble mean forecasts. These results imply that it might be difficult to improve the ensemble mean by just weighting members when an ensemble reaches a certain size. However, it is found that the weighted averaging can reduce the total forecast error more when a raw ensemble-mean forecast itself is less accurate. It is also expected that the effectiveness of weighted averaging should be improved when the ensemble spread is improved or when the ranking method itself is improved, although such an improvement should not be expected to be too big (probably less than 10%, on average).