Challenges in evaluating implementation and effectiveness in real-world settings: evaluation proposal for school-based health-promoting intervention

Abstract

Summary There are various research designs and approaches to investigate how health-promoting activities are implemented in complex, real-world systems, and to identify potential health effects that might occur following implementation. Although literature describes guidelines to perform and report about implementation research and effect evaluations, no specific guidelines exist on analysing and reporting about the combination of effectiveness data and implementation data collected as part of intervention evaluation in complex and diverse settings. This paper describes the evaluation of primary school-based health-promoting activities in complex systems. Furthermore, an approach for data categorization inspired by Rogers’ Diffusion of Innovations theory is presented that can facilitate structuring the study’s results and relating the degree of implementation to any impact on effectiveness outcomes that might be observed. Researchers interested in using this approach for data categorization have to ensure that the following three conditions are met: (i) data on an intervention’s efficacy in a controlled setting with optimal implementation is available; (ii) key points that define an intervention’s optimal implementation are available and (iii) an evaluation study is performed, collecting both effectiveness data and implementation data in a real-world context. This data categorization approach can be useful to generate more insight into an intervention’s effectiveness under varying circumstances, and optimal support and advice can be provided to stakeholders to achieve maximum impact of population-based health-promoting interventions in complex, real-world systems. However, the proposed approach is a first suggestion and further testing and adaptation is necessary to increase its usefulness. Knowledge and experience sharing among researchers performing comparable research can increase the knowledge base regarding this subject.