Immediate Post-concussion and Cognitive Testing: Ceiling Effects, Reliability, and Implications for Interpretation

Abstract

Abstract Objective The most commonly used computerized neurocognitive test in concussion assessment and management, Immediate Post-concussion and Cognitive Testing (ImPACT), has demonstrated varying and sometimes concerning levels of test–retest reliability. This study aimed to further examine ImPACT’s psychometric qualities and whether ceiling effects may suppress its reliability. Method A total of 300 consecutively selected ImPACT score reports for students attending a secondary school between 2010 and 2015 were reviewed. Test–retest reliabilities for composite scores and subscales were computed using Pearson product moment correlations (r) and intraclass correlation coefficients. To examine the potential influence of ceiling effects, we conducted frequency analyses of scores falling at, or near, the maximum possible score. Results A total of 92 score reports met inclusion criteria. Test–retest reliabilities ranged from 0.42 to 0.69 for composite scores and 0.19 to 0.71 for subscales. Subscales comprising the Verbal and Visual Memory composites evidenced the most prominent ceiling effects. Conclusions Obtained test–retest reliabilities were consistent with a large segment of the literature and add to concerns about ImPACT’s reliability. Furthermore, at least in a select sample, this study identified sizeable ceiling effects that likely diminish the reliability of some composite scores. To mitigate the risk of false-negative errors on post-injury testing, ImPACT users should be cognizant of such ceiling effects. Supplemental, or alternative, approaches to protect against underestimating baseline functioning also warrant consideration.