Semi-autonomous sounding selection for OCO-2

Abstract

Abstract. Many modern instruments generate more data than may be fully processed in a timely manner. For some atmospheric sounders, much of the raw data cannot be processed into meaningful observations due to suboptimal viewing conditions, such as the presence of clouds. Conventional solutions are quick, empirical-threshold filters hand-created by domain experts to weed out unlikely or unreasonable observations, coupled with randomized down sampling when the data volume is still too high. In this paper, we describe a method for the construction of a subsampling and ordering solution that maximizes the likelihood that a requested data subset will be usefully processed. The method can be used for any metadata-rich source and implicitly discerns informative vs. non-informative data features while still permitting user feedback into the final features selected for filter implementation. We demonstrate the method by creating a selector for the spectra of the Japanese GOSAT satellite designed to measure column averaged mixing ratios of greenhouse gases including carbon dioxide (CO2). This is done within the Atmospheric CO2 Measurements from Space (ACOS) NASA project with the intention of eventual use during the early Orbiting Carbon Observatory-2 (OCO-2) mission. OCO-2 will have a 1.5 orders of magnitude larger data volume than ACOS, requiring intelligent pre-filtration.