An Investigation of Bias in Reports of the National Assessment of Educational Progress

Abstract

This article investigates plausible explanations for the observed heterogeneity among jurisdictions in the exclusion rates of students with disabilities and English language learners in administrations of the National Assessment of Educational Progress (NAEP). It also examines the operating characteristics of a particular class of methods, called full-population estimates (FPE), that carry out statistical adjustments to NAEP’s reported scores to address the possible bias due to differential exclusion rates. The conclusions are that for many states there is a strong likelihood of bias in the results reported and that neither the current NAEP procedure nor the FPE methodologies constitute an ideal solution to the problem. Some alternative methods and related research questions are indicated. It is suggested that the general strategy employed here may be useful in investigating similar questions in other large-scale assessment surveys.