Within-Subject Comparison of Changes in a Pretest-Posttest Design

Abstract

The authors propose a method to compare the influence of a treatment on different properties within subjects. The properties are measured by several Likert-type—scaled items. The results show that many existing approaches, such as repeated measurement analysis of variance on sum and mean scores, a linear partial credit model, and a graded response model, conceptualize a comparison of changes in a way that depends on the distribution of the pretest values, but in the present article, change is measured in terms of the conditional distributions of posttest values, given the pretest values. A multivariate regression and analysis of covariance approach is unbiased but shows power deficiencies in a simulation study. The authors suggest a new approach based on poststratification (i.e., aggregating change information conditional on each pretest value), which is unbiased and has a superior power. The approach is applied in a study that compares the influence of a certain piece of music on five different basic emotions.