The Effect of Altering the Position of Options in a Multiple-Choice Examination

Abstract

Many testing programs rely on equating procedures to achieve comparability of scores on alternate test forms. One commonly accepted rule for developing equated examinations using the common-items nonequivalent groups (CINEG) design is that items common to the two examinations being equated should be identical. In test construction practice, this rule has been extended to include even the order in which options appear in the two examinations. The present study examined the performance of a common set of items on an examination in which the order of options for one test form was experimentally manipulated. The study sought to determine whether reordering multiple choice item options results in any significant effect on item difficulty. It was found that reordering options can have significant but unpredictable effects on item performance. A linkage is made to previous research and cautions are suggested regarding the effects of reordering options.