Using measurements for evaluation of black carbon modeling

Abstract

Abstract. The ever increasing use of air quality and climate model assessments to underpin economic, public health, and environmental policy decisions makes effective model evaluation critical. This paper discusses the properties of black carbon and light attenuation and absorption observations that are the key to a reliable evaluation of black carbon model and compares parametric and nonparametric statistical tools for the quantification of the agreement between models and observations. Black carbon concentrations are simulated with TM5/M7 global model from July 2002 to June 2003 at four remote sites (Alert, Jungfraujoch, Mace Head, and Trinidad Head) and two regional background sites (Bondville and Ispra). Equivalent black carbon (EBC) concentrations are calculated using light attenuation measurements from January 2000 to December 2005. Seasonal trends in the measurements are determined by fitting sinusoidal functions and the representativeness of the period simulated by the model is verified based on the scatter of the experimental values relative to the fit curves. When the resolution of the model grid is larger than 1° × 1°, it is recommended to verify that the measurement site is representative of the grid cell. For this purpose, equivalent black carbon measurements at Alert, Bondville and Trinidad Head are compared to light absorption and elemental carbon measurements performed at different sites inside the same model grid cells. Comparison of these equivalent black carbon and elemental carbon measurements indicates that uncertainties in black carbon optical properties can compromise the comparison between model and observations. During model evaluation it is important to examine the extent to which a model is able to simulate the variability in the observations over different integration periods as this will help to identify the most appropriate timescales. The agreement between model and observation is accurately described by the overlap of probability distribution (PD) curves. Simple monthly median comparisons, the Student's t-test, and the Mann-Whitney test are discussed as alternative statistical tools to evaluate the model performance. The agreement measured by the Student's t-test, when applied to the logarithm of EBC concentrations, overestimates the higher PD agreements and underestimates the lower PD agreements; the Mann-Whitney test can be employed to evaluate model performance on a relative scale when the shape of model and experimental distributions are similar.