Ensuring treatment fidelity in intervention studies: Developing a checklist and scoring system within a behaviour change paradigm

Abstract

AbstractBackgroundTreatment fidelity refers to the degree to which an intervention is implemented as intended. Promoting treatment fidelity is important to achieve a valid comparison in intervention research. However, it is often underreported: few studies detail the use and development of fidelity measures. This study aims to promote the treatment fidelity of a modified version of the Derbyshire Language Scheme (M‐DLS), a manualised intervention for children with language difficulties, by exploring participants’ opinions on training and intervention delivery. Results inform development of a checklist and scoring system to monitor and promote treatment fidelity in a comparison trial.MethodTen student speech and language therapists (SLTs) and two research assistants (RAs) participated in the study. All received training on the M‐DLS, and 10 were video‐recorded completing role‐plays of an M‐DLS session in small groups. Feedback was gathered after training and role‐plays in focus groups and interviews. Feedback was interpreted using the constructs of the Theoretical Domains Framework (TDF). A treatment fidelity checklist was then developed using the feedback. The first author and two RAs rated role‐play videos using the checklist to trial it to inform amendments and to promote interrater reliability. Interrater agreement was calculated using Spearman's test of correlation.ResultsParticipants discussed the importance of having clear materials and time to practise sessions. They suggested amendments to the materials and training to promote treatment fidelity. The checklist and scoring system accounted for participants’ suggestions, with amendments detailed in a log. Spearman's correlation results suggested agreement between the raters was strong.ConclusionsResults emphasise the importance of training quality, practice and reflective opportunities and clear materials to promote treatment fidelity. The construction of the checklist and scoring system was described in detail, informing the development of future checklists. After further trialling, the checklist can be used to ensure the M‐DLS is delivered with high treatment fidelity in the comparison trial.What this paper addsWhat is already known on this subject Treatment fidelity is an essential component of intervention effectiveness and efficacy studies, ensuring the intervention is delivered as intended. It is also an essential component of evidence-based clinical practice. However, few research studies report the treatment fidelity process or publish the checklists used, depriving clinicians of useful information for implementation.What this study adds This study describes in detail the iterative process of treatment fidelity checklist development, engaging those implementing the intervention in development. This ensured clarity and interrater reliability of the checklist. Furthermore, a novel scoring system was developed so that accuracy of implementation can be easily compared across users and across practice attempts.What are the clinical implications of this work? The importance of treatment fidelity when implementing effective and efficacious interventions cannot be overstated. The treatment fidelity checklist developed for research can be easily adopted to support accurate implementation in clinical practice through an audit process.