Characteristics of statistical parameters used to interpret least-squares results.

Abstract

Abstract This paper discusses properties of several statistical parameters that are useful in judging the quality of least-squares fits of experimental data and in interpreting least-squares results. The presentation includes simplified equations that emphasize similarities and dissimilarities among the standard error of estimate, the standard deviations of slopes and intercepts, the correlation coefficient, and the degree of correlation between the least-squares slope and intercept. The equations are used to illustrate dependencies of these parameters upon experimentally controlled variables such as the number of data points and the range and average value of the independent variable. Results are interpreted in terms of which parameters are most useful for different kinds of applications. The paper also includes a discussion of joint confidence intervals that should be used when slopes and intercepts are highly correlated and presents equations that can be used to judge the degree of correlation between these coefficients and to compute the elliptical joint confidence intervals. The parabolic confidence intervals for calibration cures are also discussed briefly.