Tracking the randomized rollout of a Veterans Affairs opioid risk management tool: A multi-method implementation evaluation using the Consolidated Framework for Implementation Research (CFIR)

Abstract

Background The Veterans Health Administration (VHA) developed the Stratification Tool for Opioid Risk Mitigation (STORM) dashboard to assist in identifying Veterans at risk for adverse opioid overdose or suicide-related events. In 2018, a policy was implemented requiring VHA facilities to complete case reviews of Veterans identified by STORM as very high risk for adverse events. Nationally, facilities were randomized in STORM implementation to four arms based on required oversight and by the timing of an increase in the number of required case reviews. To help evaluate this policy intervention, we aimed to (1) identify barriers and facilitators to implementing case reviews; (2) assess variation across the four arms; and (3) evaluate associations between facility characteristics and implementation barriers and facilitators. Method Using the Consolidated Framework for Implementation Research (CFIR), we developed a semi-structured interview guide to examine barriers to and facilitators of implementing the STORM policy. A total of 78 staff from 39 purposefully selected facilities were invited to participate in telephone interviews. Interview transcripts were coded and then organized into memos, which were rated using the −2 to + 2 CFIR rating system. Descriptive statistics were used to evaluate the mean ratings on each CFIR construct, the associations between ratings and study arm, and three facility characteristics (size, rurality, and academic detailing) associated with CFIR ratings. We used the mean CFIR rating for each site to determine which constructs differed between the sites with highest and lowest overall CFIR scores, and these constructs were described in detail. Results Two important CFIR constructs emerged as barriers to implementation: Access to knowledge and information and Evaluating and reflecting. Little time to complete the CASE reviews was a pervasive barrier. Sites with higher overall CFIR scores showed three important facilitators: Leadership engagement, Engaging, and Implementation climate. CFIR ratings were not significantly different between the four study arms, nor associated with facility characteristics. Plain Language Summary: The Veterans Health Administration (VHA) created a tool called the Stratification Tool for Opioid Risk Mitigation dashboard. This dashboard shows Veterans at risk for opioid overdose or suicide-related events. In 2018, a national policy required all VHA facilities to complete case reviews for Veterans who were at high risk for these events. To evaluate this policy implementation, 78 staff from 39 facilities were interviewed. The Consolidated Framework for Implementation Research (CFIR) implementation framework was used to create the interview. Interview transcripts were coded and organized into site memos. The site memos were rated using CFIR's −2 to +2 rating system. Ratings did not differ for four study arms related to oversight and timing. Ratings were not associated with facility characteristics. Leadership, engagement and implementation climate were the strongest facilitators for implementation. Lack of time, knowledge, and feedback were important barriers.