Implementation fidelity, student outcomes, and cost-effectiveness of train-the-trainer strategies for Masters-level therapists in urban schools: a cluster randomized trial

Abstract

Abstract Background: Little is known about the effectiveness and cost-effectiveness of train-the-trainer (TT) implementation strategies in supporting mental health evidence-based practices (EBPs) in schools, and about the optimal level of support needed for TT strategies. TT implementation strategies have been found to be effective and are increasingly being employed in large mental health systems to address the growing need for mental health services. However, very little research has been conducted using TT implementation strategies in schools. Masters-level therapists providing services in school settings need effective and cost-effective implementation strategies and TT is a promising approach to meet student mental health needs. Methods: A type 2 hybrid cluster randomized controlled trial was conducted comparing two TT implementation strategies (Train-the-Trainer [TT], and Train-the-Trainer + ongoing consultation for trainers [TT+]) on the delivery of a group cognitive behavioral treatment protocol for anxiety disorders in 22 urban schools. Participants were 33 therapists, 29 supervisors, and 125 students at risk for anxiety disorders. Implementation outcomes included implementation fidelity and treatment dosage. Student outcomes were child- and parent-reported symptoms of anxiety, child-reported symptoms of depression, and teacher-reported academic engagement. Results: There were no significant differences for implementation fidelity and treatment dosage between therapists in the TT and TT+ conditions. However, as predicted, therapists receiving both types of TT implemented the intervention with relatively high levels of implementation fidelity, and we could not be 95% confident that TT+ was cost-effective compared to TT. The cost of TT was lower than TT+ and still maintained relatively high levels of fidelity. A mixed effects modeling approach for student outcomes revealed time effects for symptoms of anxiety and depression reported by students, and emotional disaffection reported by teachers. Regardless of the TT condition, the time effects indicated an improvement from pre-treatment to post-treatment in symptoms of anxiety and depression and academic emotional engagement. There were no condition or condition x times effects. The average cost of therapist, supervisor, and consultant time required to implement the intervention in each condition were $1002 for TT and $1422 for TT+. The point estimate for the cost-effectiveness ratio indicated that TT+ increases costs by $82 per one point improvement in the content fidelity score. Conclusions: A TT implementation approach consisting of a thorough initial training workshop for therapists and supervisors as well as ongoing supervision for therapists was found to be effective for obtaining adequate levels of fidelity at a lower cost, compared to the TT+ condition that also included ongoing external expert consultation for supervisors. Trial registration: ClinicalTrials.gov identifier: NCT02651402