Combining estimators in interlaboratory studies and meta‐analyses

Abstract

AbstractMany statistical methods (estimators) are available for estimating the consensus value (or average effect) and heterogeneity variance in interlaboratory studies or meta‐analyses. These estimators are all valid because they are developed from or supported by certain statistical principles. However, no estimator can be perfect and must have error or uncertainty (known as estimator uncertainty). For a given dataset, the consensus value and heterogeneity variance given by different estimators can often differ significantly. Consequently, the choice of different estimators can affect the conclusion of an interlaboratory study or meta‐analysis. However, there is no universally accepted metric for determining which estimator is optimal among a set of candidate estimators. Instead of selecting and using a single estimator, this paper proposes an estimator‐averaging approach to combine a set of individual estimators. The final averaged estimator is a linear combination of individual estimators, which accounts for three sources of uncertainties including the estimator uncertainty. Monte Carlo simulations were performed to examine the long‐run performance of four individual estimators and the proposed averaged estimators. A case study: the determination of the Newtonian constant of gravitation is presented, where 10 individual estimators (eight frequentist weighted average methods and two Bayesian methods) are combined using the proposed estimator‐averaging approach.