How well must climate models agree with observations?

Abstract

The usefulness of a climate-model simulation cannot be inferred solely from its degree of agreement with observations. Instead, one has to consider additional factors such as internal variability, the tuning of the model, observational uncertainty, the temporal change in dominant processes or the uncertainty in the forcing. In any model-evaluation study, the impact of these limiting factors on the suitability of specific metrics must hence be examined. This can only meaningfully be done relative to a given purpose for using a model. I here generally discuss these points and substantiate their impact on model evaluation using the example of sea ice. For this example, I find that many standard metrics such as sea-ice area or volume only permit limited inferences about the shortcomings of individual models.