Testing mechanisms in large-N realistic evaluations

Abstract

The concept of generative mechanisms is central to the realistic evaluation approach. They are seen as the drivers of change. Qualitative evidence is especially well suited to unearthing how mechanisms work. However, when evaluating a large-N programme, a realist evaluation might benefit from quantitative tests of the mechanisms hypothesized in the programme theory. Despite this, quantitative tests of mechanisms are rarely applied in realistic evaluation. The purpose of this article is therefore to illustrate how widely used quantitative social science methods can be utilized to test mechanisms in realistic evaluation when evaluating large-N programmes. The proposed methods focus on intra-programme comparison based on the strength of a quantitatively measured mechanism. The article illustrates how simple statistical methods in the form of descriptive statistics and logistic regression can be used to test the influence of mechanisms in generating outcomes.