Phantom Effects in Multilevel Compositional Analysis

Abstract

This article combines statistical and applied research perspective showing problems that might arise when measurement error in multilevel compositional effects analysis is ignored. This article focuses on data where independent variables are constructed measures. Simulation studies are conducted evaluating methods that could overcome the measurement error problems. These methods are a multilevel regression model with reliability correction, a multilevel model with plausible values, and a multilevel latent regression model (doubly latent model). While the latter one performs best, all models have their advantages and disadvantages. Examples using data from the Trends in International Mathematics and Science Study are shown to present the behavior of the tested models in real-data situations and indicate the consequences of ignoring the problem of reliability.