The Reliability Analysis of Speaking Test in Computer-Assisted Language Learning (CALL) Environment

Abstract

Speaking ability is regarded as among the essential aspects of language development. Oral examination appeared challenging to evaluate due to the presence of human evaluators. The speaking method depends on the test’s reliability, determined by the raters’ scores. The current study is aimed at evaluating the speaking test’s interrater reliability utilized to measure the speaking performance of Common First Year (CFY) students during remote learning. The data were obtained from 56 EFL learners using a scoring sheet and rubrics. Eight raters were responsible for rating the study. The speaking test’s reliability was estimated using quantitative data analysis. Correlation coefficients and the Bland-Altman test were employed to assess raters’ agreement. SPSS was utilized to analyze the data in this investigation. The study’s findings suggested that the speaking exam used in the CFY program during remote learning has shown some reliability on correlations and acceptable norms on the Bland-Altman test.