A Fast Quality Control of 0.5 Hz Temperature Data in China

Abstract

Fast quality control (FQC) is important to deal with high-frequency observation records at meteorological station networks in time, and may check whether the records fall within a range of acceptable values. Threshold tests in the previous quality control methods for monthly, daily, or hourly observation data do not work well for 0.5 Hz data at a single station. In this study, we develop an algorithm for the automatic determination of maximum and minimum minute thresholds for 0.5 Hz temperature data in the data collection phase of the newly built stations. The fast threshold test based on the percentile threshold (0.1–99.9%) and standard deviation scheme is able to efficiently identify the incorrect data in the current minute. A visual graph is generated every minute, and the time series of the data records and the thresholds are displayed by the automated graphical procedures. The observations falling outside the thresholds are flagged and then a manual check is performed. This algorithm has the higher efficiency and lower computational requirement in identifying out the obvious outliers of 0.5 Hz data in real or near-real time observation. Meanwhile, this algorithm can also find problems in observation instruments. This method is applied to the quality control of 0.5 Hz data at two Tianjin experiment stations and hourly data at one Shenyang experiment station. The results show that this fast threshold test may be a viable option in the data collection phase. The advantage of this method is that the computation requires less memory and the computational burden is reduced for real or near-real time observations, so it may be extended to test other meteorological variables measured by high-frequency measurement systems.