Blocked Versus Randomized Format of Questionnaires

Abstract

The notion of item context effects implies that psychometric properties of an item or scale are altered by the presentation format, for example, blocked versus randomized. In an experimental study with high school students, the experimental group ( n = 407) answered a four-dimensional academic self-concept questionnaire, in which the items were presented in a grid with eight identical item stems forming the rows and four academic subjects (math, physics, German, English) forming the columns, thereby maximizing potential blocking effects. The control group ( n = 396) answered the identical items presented in a traditional randomized order. Multiple-group confirmatory factor analyses demonstrated similar factorial structures (i.e., factor loadings, factor variances, and factor covariances) of the two versions. Scale means, scale homogeneities, and relations with grades in the corresponding and noncorresponding subjects were very similar. Practical implications of these results for economical measurement of educational and psychological concepts (i.e., using grids) are discussed.