Further Investigation of Nonfunctioning Options in Multiple-Choice Test Items

Abstract

This study addressed the hypothesis that, after the systematic elimination of nonfunctioning options, four-option test items would perform as well as five-option test items having one or more dysfunctional distracters. The study consisted of two investigations involving an examination administered to 700 candidates for certification in a medical specialty. In the first investigation, it was found that content experts exhibited a high degree of accuracy in identifying nonfunctioning options where the criterion was empirical item analysis data. The second phase of the study compared five-option versions of multiple-choice items with four-option versions in which a nonfunctioning option had been removed. Results indicated that (a) removal of a nonfunctioning option resulted in a slight, non significant overall increase in item difficulty and no significant differences in item discrimination, (b) a test consisting of items with a nonfunctioning option removed was nearly equally reliable compared with a set of the same items in a five-option format, and (c) the use of empirical or judgmental methods of identifying nonfunctioning options was not related to changes in item performance. Implications, cautions, and suggestions for future research are provided.