Standard Errors in Educational Assessment

Abstract

In many educational settings, educational gains are measured and evaluated rather than absolute levels of achievement. Gains might be estimated for individual students,teachers, schools, districts, and so forth. In some educational programs, schools are required to make "statistically significant" progress over the course of one school year. This would typically require and estimate of the standard error (SE for short) of the gain, which is a number representing the precision of the gain similar to the "margin of error" in polls. Because SEs can be used to define educational targets, it is important to understand precisely what a standard error is -- and this requires going beyond the simple textbook definition. Statistical methods are tools for understanding social processes, but there is no necessary connection between a statistical method and an empirical outcome. A policy analyst must ask how closely features of the statistical theory correspond to aspects of the measured outcomes for a given purpose. For example, how much does it matter if the assumption of random sampling is violated in certain ways? Can one assume that the children or educators at a particular school during a given year constitute a random sample of some population that is perhaps spread across time, space, as well as cultural and institutional dimensions?