Comparison of straight line curve fit approaches for determining parameter variances and covariances

Abstract

Pressure balances are known to have a linear straight line equation of the formy = ax + bthat relates the applied pressurexto the effective areay, and recent work has investigated the use of Ordinary Least Squares (OLS), Weighted Least Squares (WLS), and Generalized Least Squares (GLS) regression schemes in order to quantify the expected values of the zero-pressure areaA0 = band distortion coefficientλ = a/bin pressure balance models of the formy = A0(1 + λx). The limitations with conventional OLS, WLS and GLS approaches is that whilst they may be used to quantify the uncertaintiesu(a) andu(b) and the covariancecov(a,b), it is technically challenging to analytically quantify the covariance termcov(A0,λ) without additional Monte Carlo simulations. In this paper, we revisit an earlier Weighted Total Least Squares with Correlation (WTLSC) algorithm to determine the variancesu2(a) andu2(b) along with the covariancecov(a,b), and develop a simple analytical approach to directly infer the corresponding covariancecov(A0,λ) for pressure metrology uncertainty analysis work. Results are compared to OLS, WLS and GLS approaches and indicate that the WTLSC approach may be preferable as it avoids the need for Monte Carlo simulations and additional numerical post-processing to fit and quantify the covariance term, and is thus simpler and more suitable for industrial metrology pressure calibration laboratories. Novel aspects is that a Gnu Octave/Matlab program for easily implementing the WTLSC algorithm to calculate parameter expected values, variances and covariances is also supplied and reported.