Will it work here? A realist approach to local decisions about implementing interventions evaluated as effective elsewhere

Abstract

There is increasing interest in what evidence is needed to inform decisions about transporting interventions evaluated as effective to new settings. There has been less attention to how local decision-makers decide whether to implement such interventions immediately or subject to further evaluation. Using the example of school-based social and emotional learning, we consider this drawing on realist methods. We suggest decisions need to assess existing evaluations not merely in terms of whether the intervention was effective but also: how the intervention was implemented and what contextual factors affected this (drawing on process evaluation); and for whom the intervention was effective and through what mechanisms (drawing on mediation, moderation and qualitative comparative analyses from primary studies and/or systematic reviews). We contribute new insights to local needs assessments, suggesting that these should assess: the potential, capability, contribution and capacity present in the new setting for implementation; and whether similar ‘aetiological mechanisms’ underlie adverse outcomes locally as in previous evaluations. We recommend that where there is uncertainty concerning whether an intervention can feasibly be implemented this indicates the need for piloting of implementation. Where there is uncertainty concerning whether implementation of the intervention will trigger intended mechanisms, this suggests the need for a new effectiveness trial. Where there is uncertainty concerning whether intervention mechanisms, even if triggered, will generate the intended outcomes, this suggests that decision-makers may need to look to other types of intervention as being needed for their setting instead.