The Relationship of Expert-System Scored Constrained Free-Response Items to Multiple-Choice and Open-Ended Items

Abstract

This study examined the relationship of an expert- system scored constrained free-response item (requir ing the student to debug a faulty computer program) to two other item types: (1) multiple-choice and (2) free- response (requiring production of a program). Confir matory factor analysis was used to test the fit of a three-factor model to these data and to compare the fit of the model to three alternatives. These models were fit using two random-half samples, one given a faulty program containing one bug and the other a program with three bugs. A single-factor model best fit the data for the sample taking the one-bug constrained free re sponse and a two-factor model fit the data somewhat better for the second sample. In addition, the factor intercorrelations showed this item type to be highly re lated to both the free-response and multiple-choice measures.