Where Are the Costs? Using an Economic Analysis of Educational Interventions Approach to Improve the Evaluation of a Regional School Improvement Programme

Abstract

Education systems are moving to a more evidence-informed paradigm to improve outcomes for learners. To help this journey to evidence, robust qualitative and quantitative research can help decisionmakers identify more promising approaches that provide value for money. In the context of the utilisation of scarce resources, an important source of evidence commonly used in health and social care research is an understanding of the economic impact of intervention choices. However, there are currently very few examples where these methodologies have been used to improve the evaluation of education interventions. In this paper we describe the novel use of an economic analysis of educational interventions (EAEI) approach to understand both the impact and the cost of activities in the evaluation of a formative assessment implementation project (FAIP) designed to improve teachers’ understanding and use of formative assessment strategies. In addition to utilizing a mixed method quasi-experimental design to explore the impact on learner wellbeing, health utility and attainment, we describe the use of cost-consequence analysis (CCA) to help decisionmakers understand the outcomes in the context of the resource costs that are a crucial element of robust evaluations. We also discuss the challenges of evaluating large-scale, universal educational interventions, including consideration of the economic tools needed to improve the quality and robustness of these evaluations. Finally, we discuss the importance of triangulating economic findings alongside other quantitative and qualitative information to help decisionmakers identify more promising approaches based on a wider range of useful information. We conclude with recommendations for more routinely including economic costs in education research, including the need for further work to improve the utility of economic methods.