Assessing the Skill of Yes/No Forecasts for Markov Observations

Abstract

Abstract Briggs and Ruppert recently introduced a new, easy-to-calculate economic skill/value score for use in yes/no forecast decisions, of which precipitation forecast decisions are an example. The advantage of this new skill/value score is that the sampling distribution is known, which allows one to perform hypothesis tests on collections of forecasts and to say whether a given skill/value score is significant or not. Here, the climate skill/value score is taken and extended to the case where the predicted series is first-order Markov in nature, of which, again, precipitation occurrence series can be an example. It is shown that, in general, Markov skill/value is different and more demanding than is persistence skill. Persistence skill is defined as improvement over forecasts that state that the next value in a series will equal the present value. It is also shown that any naive forecasts based solely on the Markov parameters is always at least as skillful/valuable as are persistence forecasts; in general, persistence forecasts should not be used. The distribution for the Markov skill score is presented, and examples of hypothesis testing for precipitation forecasts are given. These skill scores are graphed for a wide range of forecast/user loss functions, a process that makes their interpretation simple.