Constructing Items Measuring Logical Operational Thinking: Facet Design-Based Item Construction Using Multiple Categories Scoring

Abstract

The development and application of test items of logical operational thinking are described. Twelve classical Piagetian experiments were transformed into objective test items using Guttman's facet design. In the item construction the response options received such a theoretical underpinning and form that they were considered to represent in a prototypical way all three operational levels, allowing interpretation of hypothetical reasoning strategies behind the student's choice of alternative. The items in multiple-choice format were given to four samples of Swedish students as a national option of the Second IEA Science Study: 1322 students in Grade 3, 1423 in Grade 4, 784 in Grade 7, and 725 in Grade 8. Twenty-two items were administered, of which 5 were specific for Grades 3 and 4, and 5 specific for Grades 7 and 8. The Rasch partial credit model was applied in order to take into account the information in all response options. Fifteen items in Grades 3 and 4, and 18 items in Grades 7 and 8 showed acceptable fit to the measurement model. The overall mean-square fit statistics were in accordance with model expectations.