Quantitation of measurement error with Optimal Segments: basis for adaptive time course smoothing

Abstract

We introduce a novel technique for estimating measurement error in time courses and other continuous curves. This error estimate is used to reconstruct the original (error-free) curve. The measurement error of the data is initially assumed, and the data are smoothed with "Optimal Segments" such that the smooth curve misses the data points by an average amount consistent with the assumed measurement error. Thus the differences between the smooth curve and the data points (the residuals) are tentatively assumed to represent the measurement error. This assumption is checked by testing the residuals for randomness. If the residuals are nonrandom, it is concluded that they do not resemble measurement error, and a new measurement error is assumed. This process continues reiteratively until a satisfactory (i.e., random) group of residuals is obtained. In this case the corresponding smooth curve is taken to represent the original curve. Monte Carlo simulations of selected typical situations demonstrated that this new method ("OOPSEG") estimates measurement error accurately and consistently in 30- and 15-point time courses (r = 0.91 and 0.78, respectively). Moreover, smooth curves calculated by OOPSEG were shown to accurately recreate (predict) original, error-free curves for a wide range of measurement errors (2-20%). We suggest that the ability to calculate measurement error and reconstruct the error-free shape of data curves has wide applicability in data analysis and experimental design.