The Relationship between Two Methods of Evaluating an Examinee's Difference Scores

Abstract

The “nonoverlapping intervals” and “reliable difference” approaches for assessing difference scores are compared and shown to be consistent alternatives when the proper z is used to construct confidence intervals. Formulas for computing the probabilities of correct interpretation (power), overinterpretation, misinterpretation, and underinterpretation for four popular confidence interval approaches and the reliable difference approach are presented. The probability formulas show that the intuitive inference concerning the statistical significance level of nonoverlapping intervals is incorrect. The limitations of the nonoverlapping intervals approach in applied situations are discussed. It appears that in most situations the reliable difference is the easiest to apply.