Challenges, Opportunities, and Methods for Large-Scale Evaluations

Abstract

The logic, theory, and practice of large-scale evaluation were once limited to large federal initiatives. However, with the advent of regularly collected performance measures and the often multisite implementation of quality improvement efforts, there is an opportunity to adopt large-scale evaluation methods in local and regional evaluation efforts. While ineffective programs show little variation in their ineffectiveness, effective programs generally show a range of effects. A central task of large-scale evaluation is to describe and ascribe why the same program, implemented in multiple settings, produces different effects. By its attention to variation attributable to setting, activities, outputs, and by participants and documenting the conditions in which programs achieve greater and lesser success, large-scale evaluation supports the needs of decision-makers when choosing to implement an evidence-based program. In addition to knowing a program is effective, decision-makers want to know whether it is appropriate for their situation and what facilitates or impedes effective implementation and bears on the program’s ultimate effectiveness. This article presents the different methods and approaches appropriate for effectively and efficiently constructing and executing a large-scale evaluation that will provide decision-makers the evidence they need for evidence-informed adoption of effective programs.