Optimal Detection of Certain Forms of Inappropriate Test Scores

Abstract

Optimal appropriateness indices, recently introduced by Levine and Drasgow (1984), provide the highest rates of detection of aberrant response patterns that can be obtained from item responses. In this article they are used to study three important problems in ap propriateness measurement. First, the maximum detec tion rates of two particular forms of aberrance are de termined for a long unidimensional test. These detection rates are shown to be moderately high. Sec ond, two versions of the standardized l0 appropriate ness index are compared to optimal indices. At low false alarm rates, one standardized l0 index has detec tion rates that are about 65% as large as optimal for spuriously high (cheating) test scores. However, for the spuriously low scores expected from persons with ill-advised testing strategies or reading problems, both standardized l0 indices are far from optimal. Finally, detection rates for polychotomous and dichotomous scorings of the item responses are compared. It is shown that dichotomous scoring causes serious de creases in the detectability of some aberrant response patterns. Consequently, appropriateness measurement constitutes one practical testing problem in which sig nificant gains result from the use of a polychotomous item response model.