Evidence on “What Works”: An Argument for Extended-Term Mixed-Method (ETMM) Evaluation Designs

Abstract

Federal policy tools for gathering evidence on “What Works” in education, such as the What Works Clearinghouse’s (WWC) standards, emphasize randomized field trials as the preferred method for generating scientific evidence on the effectiveness of educational programs. This article argues instead for extended-term mixed-method (ETMM) designs. Emphasizing the need to consider temporal factors in gaining thorough understandings of programs as they take hold in organizational or community settings, the article asserts that formal study of contextual and site-specific variables with multiple research methods is a necessary prerequisite to designing sound field experiments for making generalized causal inferences. A theoretical rationale and five guiding principles for ETMM designs are presented, with suggested revisions to the WWC’s standards.