Applying RE-AIM to evaluations of Veterans Health Administration Enterprise-Wide Initiatives: lessons learned

Abstract

IntroductionThe United States Veterans Health Administration (VHA) Office of Rural Health funds Enterprise-Wide Initiatives (system-wide initiatives) to spread promising practices to rural Veterans. The Office requires that evaluations of Enterprise-Wide Initiatives use the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. This presents a unique opportunity to understand the experience of using RE-AIM across a series of evaluations. The authors conducted a study to document the benefits and pitfalls of using RE-AIM, capture the variety of ways that the team captured the elements of RE-AIM, and develop recommendations for the future use of RE-AIM in evaluation.Materials and methodsThe authors first conducted a document review to capture pre-existing information about how RE-AIM was used. They subsequently facilitated two focus groups to gather more detailed information from team members who had used RE-AIM. Finally, they used member-checking throughout the writing process to ensure accurate data representation and interpretation and to gather additional feedback.ResultsFour themes emerged from the document review, focus groups, and member checking. RE-AIM: provides parameters and controls the evaluation scope, “buckets” are logical, plays well with other frameworks, and can foster collaboration or silo within a team. Challenges and attributes for each RE-AIM dimension were also described.DiscussionOverall, participants reported both strengths and challenges to using RE-AIM as an evaluation framework. The overarching theme around the challenges with RE-AIM dimensions was the importance of context. Many of these benefits and challenges of using RE-AIM may not be unique to RE-AIM and would likely occur when using any prescribed framework. The participants reported on the RE-AIM domains in a variety of ways in their evaluation reports and were not always able capture data as originally planned. Recommendations included: start with an evaluation framework (or frameworks) and revisit it throughout the evaluation, consider applying RE-AIM PRISM (Practical Robust Implementation Framework) to gain a broader perspective, and intentionally integrate quantitative and qualitative team members, regardless of the framework used.