On the Equivalence of Constructed- Response and Multiple-Choice Tests

Abstract

Two sets of mathematical reasoning and two sets of verbal comprehension items were cast into each of three formats—constructed response, standard multiple-choice, and Coombs multiple- choice—in order to assess whether tests with iden tical content but different formats measure the same attribute, except for possible differences in error variance and scaling factors. The resulting 12 tests were administered to 199 eighth-grade stu dents. The hypothesis of equivalent measures was rejected for only two comparisons: the con structed-response measure of verbal comprehen sion was different from both the standard and the Coombs multiple-choice measures of this ability. Maximum likelihood factor analysis confirmed the hypothesis that a five-factor structure will give a satisfactory account of the common variance among the 12 tests. As expected, the two major factors were mathematical reasoning and verbal comprehension. Contrary to expectation, only one of the other three factors bore a (weak) resem blance to a format factor. Tests marking the abili ty to follow directions, recall and recognition memory, and risk-taking were included, but these variables did not correlate as expected with the three minor factors.