Commencement of and Retention in Web-Based Interventions and Response to Prompts and Reminders: Longitudinal Observational Study Based on Two Randomized Controlled Trials

Abstract

Background Web-based interventions are effective for several psychological problems. However, recruitment, adherence, and missing data are challenges when evaluating these interventions. Objective This study aimed to describe the use patterns during the commencement phase, possible retention patterns (continuation of data provision), and responses to prompts and reminders among participants in 2 randomized controlled trials (RCTs) evaluating web-based interventions. Methods Data on use patterns logged in 2 RCTs aiming to reduce symptoms of anxiety and depression among adult patients recently diagnosed with cancer (AdultCan RCT) and patients with a recent myocardial infarction (Heart RCT) were analyzed. The web-based intervention in the AdultCan trial consisted of unguided self-help and psychoeducation and that in the Heart trial consisted of therapist-supported cognitive behavioral therapy. In total, 2360 participants’ use patterns at first log-in, including data collection at baseline (ie, commencement) and at 2 follow-ups, were analyzed. Both the intervention and comparison groups were analyzed. Results At commencement, 70.85% (909/1283) and 86.82% (935/1077) of the participants in AdultCan and Heart RCTs, respectively, logged in and completed baseline data collection after receiving a welcome email with log-in credentials. The median duration of the first log-in was 44 minutes and 38 minutes in AdultCan and Heart RCTs, respectively. Slightly less than half of the participants’ first log-ins were completed outside standard office hours. More than 80% (92/114 and 103/111) of the participants in both trials explored the intervention within 2 weeks of being randomized to the treatment group, with a median duration of 7 minutes and 47 minutes in AdultCan and Heart RCTs, respectively. There was a significant association between intervention exploration time during the first 2 weeks and retention in the Heart trial but not in the AdultCan trial. However, the control group was most likely to retain and provide complete follow-up data. Across the 3 time points of data collection explored in this study, the proportion of participants responding to all questionnaires within 1 week from the prompt, without a reminder, varied between 35.45% (413/1165) and 66.3% (112/169). After 2 reminders, up to 97.6% (165/169) of the participants responded. Conclusions Most participants in both RCTs completed the baseline questionnaires within 1 week of receiving the welcome email. Approximately half of them answered questions at baseline data collection outside office hours, suggesting that the time flexibility inherent in web-based interventions contributes to commencement and use. In contrast to what was expected, the intervention groups generally had lower completion rates than the comparison groups. About half of the participants completed the questionnaires without a reminder, but thereafter, reminders contributed to both baseline and follow-up retention, suggesting they were effective. Strategies to increase commencement of and retention in eHealth interventions are important for the future development of effective interventions and relevant research.