Comparison of Automatic Speech Recognition System for School-aged Children’s Narratives: Naver Clova Speech and Google Speech-to-Text

Abstract

Objectives: Language sample analysis (LSA) is a critical component of child language assessment. However, most clinicians consider LSA to be time consuming work. In particular, transcription is seen as an overwhelming task. Due to rapid technological advances, various automatic speech recognition systems have been developed. This study aimed to investigate the accuracy and the characteristics of two automatic speech recognition programs, Naver Clova Speech (Naver Clova) and Google Speech-to-Text (STT).Methods: A total of 40 school-aged children with typical development (TD) and children with language learning disabilities (LLD) participated in the study. Each child was asked to generate two fictional narratives. In total, 72 narratives produced by 36 children were used. To examine the accuracy of Naver Clova and Google STT, syllable error rate was analyzed and compared to reference transcripts. For the detailed analysis, types of error such as substitution, deletion and insertion were examined.Results: Results showed that Naver Clova was significantly lower than Google STT in error rate of transcription. But the transcription error rate of the two child groups was not significantly different. Additionally, the Naver Clova error rate was higher in substitution, deletion, and insertion respectively. The Google STT error rate, on the other hand, was higher in deletion, substitution and insertion respectively.Conclusion: Naver Clova were more accurate than Google STT in transcribing children’s narratives. But the transcription accuracy of two child groups was not different. This suggests that recently developed automatic speech recognition systems have clinical utility. These systems can reduce clinician’s workload in regards to LSA and this would contribute to qualitatively enhanced language assessment.