Using Mixed-Effects Models In Reliability Generalization Studies

Abstract

Traditionally, reliability generalization (RG) studies have used some form of the regression model to summarize score reliability of a measure across samples and examine conditions under which the reliability varies. Oftentimes, the assumptions underlying the use of multiple regression are not satisfied in RG studies. This article describes how the assumptions have been violated and introduces a more sophisticated technique, mixed-effects modeling, that can overcome many of the shortcomings of traditional approaches. A nontechnical introduction of mixed-effects models in the context of RG studies is provided along with an example using internal consistency reliability coefficients from the Beck Depression Inventory that compares results under the mixed-versus the fixed-effects models.