Evaluating agency-led adaptions to an evidence-based lifestyle intervention for adults with serious mental illness

Abstract

Background: Limited empirical evidence exists on the impact of adaptations that occur in implementing evidence-based practices (EBPs) in real-world practice settings. The purpose of this study was to measure and evaluate adaptations to an EBP (InSHAPE) for obesity in persons with serious mental illness in a national implementation in mental health care settings. Methods: We conducted telephone interviews with InSHAPE provider teams at 37 (95%) of 39 study sites during 24-month follow-up of a cluster randomized trial of implementation strategies for InSHAPE at behavioral health organizations. Our team rated adaptations as fidelity-consistent or fidelity-inconsistent. Multilevel regression models were used to estimate the relationship between adaptations and implementation and participant outcomes. Results: Of 37 sites interviewed, 28 sites (76%) made adaptations to InSHAPE ( M = 2.1, SD = 1.3). Sixteen sites (43%) made fidelity-consistent adaptations, while 22 (60%) made fidelity-inconsistent adaptations. The number of fidelity-inconsistent adaptations was negatively associated with InSHAPE fidelity scores (β = −4.29; p < .05). A greater number of adaptations were associated with significantly higher odds of participant-level cardiovascular risk reduction (odds ratio [ OR] = 1.40; confidence interval [CI] = [1.08, 1.80]; p < .05). With respect to the type of adaptation, we found a significant positive association between the number of fidelity-inconsistent adaptations and cardiovascular risk reduction ( OR = 1.59; CI = [1.01, 2.51]; p < .05). This was largely explained by the fidelity-inconsistent adaptation of holding exercise sessions at the mental health agency versus a fitness facility in the community (a core form of InSHAPE) ( OR = 2.52; 95% CI = [1.11, 5.70]; p < .05). Conclusions: This research suggests that adaptations to an evidence-based lifestyle program were common during implementation in real-world mental health practice settings even when fidelity was monitored and reinforced through implementation interventions. Results suggest that adaptations, including those that are fidelity-inconsistent, can be positively associated with improved participant outcomes when they provide a potential practical advantage while maintaining the core function of the intervention. Plain language abstract: Treatments that have been proven to work in research studies are not always one-size-fits-all. In real-world clinical settings where people receive mental health care, sometimes there are good reasons to change certain things about a treatment. For example, a particular treatment might not fit well in a specific clinic or cultural context, or it might not meet the needs of specific patient groups. We studied adaptations to an evidence-based practice (InSHAPE) targeting obesity in persons with serious mental illness made by teams implementing the program in routine mental health care settings. We learned that adaptations to InSHAPE were common, and that an adaptation that model experts initially viewed as inconsistent with fidelity to the model turned out to have a positive impact on participant health outcomes. The results of this study may encourage researchers and model experts to work collaboratively with mental health agencies and clinicians implementing evidence-based practices to consider allowing for and guiding adaptations that provide a potential practical advantage while maintaining the core purpose of the intervention.