Estimating Reliability with Small Samples: Increased Precision with Averaged Correlations.

Abstract

Both analytic and Monte Carlo methods were used to show that empirical estimates of the standard error of an averaged correlation agree quite well with theoretical expressions derived from Pearson's asymptotic equations. Where discrepancies did exist, particularly with small sample sizes, the theoretical formula slightly overestimated the actual (empirical) standard error, thus making a test of significance conservative. It was shown that by testing repeatedly with a small sample and averaging the resulting relations, one may obtain precision equivalent to single estimates from much larger samples. Furthermore, the greatest increase in precision occurred when the population correlation was small, the situation where greatest precision is needed. These results should prove of value for human factors evaluations, which often have limited subject resources.