Reliability of Scoring Telehealth Speech Sound Assessments Administered in Real-World Scenarios

Abstract

Purpose: COVID-19 caused a worldwide conversion from in-person therapy to telehealth; however, limited evidence to support the efficacy of remotely delivering standardized tests puts the future of widespread telehealth use at risk. The aim of this study is to investigate the reliability of scoring a speech sound assessment administered in real-world scenarios including two examples of telehealth technology. Method: A total of thirty-nine 3- to 8-year-olds were administered the Goldman-Fristoe Test of Articulation–Third Edition. Licensed speech-language pathologists (SLPs) concurrently scored children's responses in person and in two telehealth conditions considered typical and enhanced. Mean standard scores and interrater reliability results were compared among the three conditions. Descriptive statistics were used to summarize the frequency of technology and behavior disruptions during administration and the results of an SLP telehealth perception survey. Results: All scoring conditions were found to be highly correlated, with mean differences revealing no systematic differences of one condition over- or underestimating another. Although response agreement was high (85%–87%), final sounds in words or sounds that are difficult to observe tended to attenuate reliability. Neither child nor technology disruptions affected SLPs' ability to score responses. Despite no significant differences between conditions on scoring reliability, SLP participants reported they continued to prefer in-person over a telehealth speech sound assessment. Conclusions: This study supports the provision of a pediatric speech sound assessment using consumer-grade equipment, as in-person, typical telehealth, and enhanced telehealth scoring conditions produced similar results. However, SLP participants' skeptical attitudes toward remote delivery of standardized tests reveal an ongoing barrier to widespread telehealth use. Supplemental Material: https://doi.org/10.23641/asha.19593367