Mapping theories, models and frameworks to implement or evaluate digital health interventions: A scoping review (Preprint)

Abstract

Most research focuses on evaluating the effects of the digital health interventions (i.e. whether they work, why, and how) or understanding how best to implement them (i.e. intervention adoption and sustainment in real-world settings). The application of theories, models and frameworks (TMF) can facilitate the systematic understanding and explanation of the complex interactions between users, practices, technology, and health system factors that underpin research questions. There remains a gap in our understanding of how TMFs have been applied to guide the implementation and evaluation of digital health interventions with the real-world health system operations.

To describe and synthesize TMFs used to guide the implementation or evaluation of digital health interventions. The objectives are to (1) describe the TMFs and the constructs they target; and (2) identify how they have been used in primary studies i.e. the role(s) of those theoretical approaches. After commencing this work, we added a third post-hoc objective to reflect on the relevance and utility of our findings for knowledge user partners engaged as part of an integrated knowledge translation approach.

This scoping review was conducted in partnership with knowledge users using an integrated knowledge translation approach. We included papers (e.g., reports, empirical quantitative, qualitative, and mixed-methods studies, conference proceedings, dissertation) if primary insights resulting from the application of TMFs were presented. Any types of digital health interventions were eligible. Papers were mainly identified from 5 databases.

A total of 156 studies published between 2000 and 2022 were included. Sixty-eight distinct TMFs were identified across 85 individual studies. The six prevailing TMFs are: Consolidated Framework for Implementation Research, the Reach, Effectiveness, Adoption, Implementation, and Maintenance Framework, the Technology of Acceptance Model, the Unified Theory on Acceptance and Use of Technology, the Diffusion of Innovation Theory, and Normalization Process Theory. The most common intended roles of the six TMFs was to inform data collection (n= 86), to inform data analysis (n=69), and to identify key constructs that may serve as barriers and facilitators (n=52). The knowledge users reflected that most of the prevailing TMFs identified in this scoping review were not familiar to them, highlighting a gap between academic literature and practice. They were more familiar with Benefits Evaluation Framework and the Quadruple Aim.

As TMFs are most often applied to support data collection and analysis, researchers should consider more clearly synthesizing key insights as practical use cases to both increase relevance and digestibility of their findings. What was once an opportunity to develop a standardized reporting structure inclusive of digital health interventions function, setting, target user(s), and intended outcomes is quickly becoming an imperative in order to ensure ongoing technology transformation efforts are evidence-informed rather than anecdotally driven.