‘Improvement focused’ evaluation of place-based initiatives: An examination of three methodologies

Abstract

Place-based initiatives offer a comprehensive, whole-of-community approach to solving complex problems. Impact evaluation of complex initiatives is challenging and alternative ‘improvement focused’ methodologies, such as developmental evaluation, action research and quality improvement, are being used. Limited understanding exists about how these methodologies work when used in place-based initiatives, which contexts they are individually best suited to and what they can achieve. This article examines the methodologies of developmental evaluation, action research and quality improvement when applied to the evaluation of place-based initiatives. The approach used a realist evaluation methodology, involving a 10-year literature review and three ‘instrumental’ case studies, which is described in detail in an accompanying paper (Heery, Naccarella & McKenzie, 2018). Contextual factors, mechanisms and outcomes for the application of developmental evaluation, action research and quality improvement to place-based evaluation were identified so as to build a theory for each methodology. The three methodologies have similar mechanisms; they are cyclical and comprise planning, doing, studying and acting, and all result in continuous improvement of the initiatives, increasing collaboration and increasing evaluation capacity. A key difference between the methodologies is their perceived purpose. These findings can support practitioners, commissioners and users of evaluation working in the place-based arena in three ways: advocating for the adoption of ‘improvement focused’ evaluation; selecting the most appropriate ‘improvement focused’ methodology; and identifying and addressing the facilitating factors particular to the selected methodology to increase the effectiveness of the evaluation.