Exploring the relationships between ASS indices and CAF and the impact on Chinese college students’ oral English performance

Abstract

AbstractAutomatic Speech Scoring (ASS) has increasingly become a useful tool in oral proficiency testing for Second Language (L2) learners. However, limited studies investigate the alignment of ASS indices with the Complexity, Accuracy, and Fluency (CAF)—the three dimensions in evaluating L2 speakers’ oral proficiency, and the subsequent impact indices on the oral performance of Chinese college students. To bridge this gap, this study used comparative analysis, Pearson analysis, and linear regression analysis to delve into the relationship and correlations between paired ASS indicators of “pronunciation”, “fluency”, “integrity”, “speed”, “duration”, and “overall”, while also analyzing the relationships between “overall” and other variables. These analyses were conducted using 956 audio clips of freshmen who took the College English Test-Spoken English Test Band 4 (CET-SET-4) in May 2022 in China. The findings reveal that (1) the ASS indicators and evaluation methods are similar but not identical to those employed in prior studies; (2) “pronunciation” encapsulates both the accuracy and fluency dimensions of CAF; (3) “pronunciation” and “integrity” have significant impacts on Chinese college students’ oral English performance in read-aloud tasks. The study suggests that future research should further investigate the specific pronunciation challenges faced by Chinese college students, such as phonetics, stress, and intonation. Additionally, it highlights the need to comprehend teachers’ attitudes and preferences towards ASS to enhance its effectiveness in assessing second language (L2) learners’ oral proficiency. The study would provide some references to teachers for oral English teaching design and students for their self-assessment of oral English proficiencies.