Challenges with the Use of Digital Sham: A Systematic Review and Recommendations (Preprint)

Abstract

Digital therapeutics (DTx) are software-based products that prevent, manage, or treat a medical condition, and are delivered through a smartphone app, web application or wearable. Clinical trials assessing DTx pose challenges, foremost amongst which is designing appropriate ‘digital shams’ (or ‘digital placebos’), which should ideally mimic the DTx (in terms of design, components and duration of treatment) while omitting the active principle/component. To understand how digital shams are being used in clinical research on DTx, we conducted a systematic literature review (SLR) of DTx in neuroscience (including neurodevelopmental, neurodegenerative, and psychiatric disorders) with a focus on controlled clinical trials involving digital shams. Our search criteria identified 461 neuroscience studies involving 213 unique DTx. Most DTx were extended reality-based (40.4%) or mobile device-based (26.3%); 313 were comparative, of which 68 used shams. The most common therapeutic areas assessed in these studies were stroke (19.7%), depression (15%) and anxiety (11.3%); and the most common treatments were cognitive behavioral therapy/behavioral therapy (32.4%), physical rehabilitation (28.2%), and cognitive training (19.2%). Results: We identified the following important issues related to the use of digital shams: shams were not validated prior to use in studies; they varied widely in design; (from being near-identical to the DTx to using different software programs altogether); the level of patient engagement/satisfaction with the sham and the impact of the sham on study outcomes was infrequently reported. Conclusions: Digital shams are critical to the clinical development of DTx. Given the importance of sham controls in evaluating DTx efficacy, we provide recommendations on the key information that should be reported in a well-designed DTx trial and propose an algorithm to allow correct interpretation of DTx study results. Sham-controlled studies should be routinely used in DTx trials – in early phase studies, to help identify DTx active components, and in late-phase studies, to confirm the efficacy of DTx. Use of shams early in development will ensure that the appropriate sham control is used in later confirmatory trials.