How Sampling Errors in Covariance Estimates Cause Bias in the Kalman Gain and Impact Ensemble Data Assimilation

Abstract

Abstract Localization is the key component to the successful application of ensemble data assimilation (DA) to high-dimensional problems in the geosciences. We study the impact of sampling error and its amelioration through localization using both analytical development and numerical experiments. Specifically, we show how sampling error in covariance estimates accumulates and spreads throughout the entire domain during the computation of the Kalman gain. This results in a bias, which is the dominant issue in unlocalized ensemble DA, and, surprisingly, we find that it depends directly on the number of independent observations but only indirectly on the state dimension. Our derivations and experiments further make it clear that an important aspect of localization is a significant reduction of bias in the Kalman gain, which in turn leads to an increased accuracy of ensemble DA. We illustrate our findings on a variety of simplified linear and nonlinear test problems, including a cycling ensemble Kalman filter applied to the Lorenz-96 model. Significance Statement The dampening of long-range correlations has been the key to the success of ensemble data assimilation in global numerical weather prediction. In this paper, we show how noise in covariance estimates propagates through the state estimation process and corrupts state estimates. We show that this noise results in a bias and that this bias depends on the number of observations and not, as might be expected, on the state dimension. We go on to show how dampening long-range covariances through a process referred to as “localization” helps to mitigate the detrimental effects of this sampling noise.