Controlling for Placebo Effects in Computerized Cognitive Training Studies With Healthy Older Adults From 2016-2018: Systematic Review (Preprint)

Abstract

Computerized cognitive training has been proposed as a potential solution to age-related cognitive decline. However, published findings from evaluation studies of cognitive training games, including metastudies and systematic reviews, provide evidence both for and against transferability from trained tasks to untrained cognitive ability. There continues to be no consensus on this issue from the scientific community. Some researchers have proposed that the number of results supporting the efficacy of cognitive training may be inflated due to placebo effects. It has been suggested that placebo effects need to be better controlled by using an active control and measuring participant expectations for improvement in outcome measures.

This review examined placebo control methodology for recent evaluation studies of computerized cognitive training programs with older adult subjects, specifically looking for the use of an active control and measurement of expectations.

Data were extracted from PubMed. Evaluation studies of computerized cognitive training with older adult subjects (age ≥50 years) published between 2016 and 2018 were included. Methods sections of studies were searched for (1) control type (active or passive) and subtype (active: active-ingredient or similar-form; passive: no-contact or passive-task); (2) if expectations were measured, how were they measured, and whether they were used in analysis; and (3) whether researchers acknowledged a lack of active control and lack of expectation measurement as limitations (where appropriate).

Of the 19 eligible studies, 4 (21%) measured expectations, and 9 (47%) included an active control condition, all of which were of the similar-form type. The majority of the studies (10/19, 53%) used only a passive control. Of the 9 studies that found results supporting the efficacy of cognitive training, 5 were for far transfer effects. Regarding the limitations, due to practical considerations, the search was limited to one source (PubMed) and to search results only. The search terms may have been too restrictive. Recruitment methods were not analyzed, although this aspect of research may play a critical role in systematically forming groups with different expectations for improvement. The population was limited to healthy older adults, while evaluation studies include other populations and cognitive training types, which may exhibit better or worse placebo control than the studies examined in this review.

Poor placebo control was present in 47% (9/19) of the reviewed studies; however, the studies still published results supporting the effectiveness of cognitive training programs. Of these positive results, 5 were for far transfer effects, which form the basis for broad claims by cognitive training game makers about the scientific validity of their product. For a minimum level of placebo control, future evaluation studies should use a similar-form active control and administer a questionnaire to participants at the end of the training period about their own perceptions of improvement. Researchers are encouraged to think of more methods for the valid measure of expectations at other time points in the training.