Identifying Negatively Discriminating Items When Test Scores are Not Normally Distributed

Abstract

Traditional item analysis procedures for norm referenced testing have often included the selection of the upper and lower 27% groups for performing item discriminations. Such procedures are based upon assumptions of (a) normality, (b) score independence in the tails, and (c) equality of standard deviations in the tails of the criterion distribution. However, these characteristics may not apply to the distributions of typical classroom examinations, which tend to be negatively skewed. This study used Monte Carlo techniques to examine the power of Brennan's (1972) B to detect negatively discriminating items drawn from a variety of nonnormal population distributions. In general, it was found that power is lost when choosing equal-sized groups from asymmetric distributions. A simplified procedure for conducting an item discrimination analysis on typical classroom objective examinations is offered.