Automated assessment of second language comprehensibility: Review, training, validation, and generalization studies

Author:

Saito KazuyaORCID,Macmillan Konstantinos,Kachlicka MagdalenaORCID,Kunihara Takuya,Minematsu Nobuaki

Abstract

AbstractWhereas many scholars have emphasized the relative importance of comprehensibility as an ecologically valid goal for L2 speech training, testing, and development, eliciting listeners’ judgments is time-consuming. Following calls for research on more efficient L2 speech rating methods in applied linguistics, and growing attention toward using machine learning on spontaneous unscripted speech in speech engineering, the current study examined the possibility of establishing quick and reliable automated comprehensibility assessments. Orchestrating a set of phonological (maximum posterior probabilities and gaps between L1 and L2 speech), prosodic (pitch and intensity variation), and temporal measures (articulation rate, pause frequency), the regression model significantly predicted how naïve listeners intuitively judged low, mid, high, and nativelike comprehensibility among 100 L1 and L2 speakers’ picture descriptions. The strength of the correlation (r = .823 for machine vs. human ratings) was comparable to naïve listeners’ interrater agreement (r = .760 for humans vs. humans). The findings were successfully replicated when the model was applied to a new dataset of 45 L1 and L2 speakers (r = .827) and tested under a more freely constructed interview task condition (r = .809).

Funder

Economic and Social Research Council

Leverhulme Trust

Spencer Foundation

Publisher

Cambridge University Press (CUP)

Subject

Linguistics and Language,Language and Linguistics,Education

Reference67 articles.

1. Conceptual and empirical relationships between temporal measures of fluency and oral English proficiency with implications for automated scoring

2. Age Effects in Second Language Learning: Stepping Stones Toward Better Understanding

3. Povey, D. , Ghoshal, A. , Boulianne, G. , Burget, L. , Glembek, O. , Goel, N. , Hannemann, M. , Motlicek, P. , Qian, Y. , Schwarz, P. , Silovsky, J. , Stemmer, G. , & Vesely, K. (2011). The Kaldi speech recognition toolkit. Proceedings of the IEEE 2011 Workshop on Automatic Speech Recognition and Understanding, Hilton Waikoloa Village, Big Island, Hawaii, US. https://infoscience.epfl.ch/record/192584

4. Experience Effects on the Development of Late Second Language Learners’ Oral Proficiency

Cited by 4 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

1. A New Frontier in AI-Assisted English Oral Presentation Assessment;2023 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE);2023-11-28

2. The Effects of Task Repetition Schedules on L2 Fluency Enhancement;Languages;2023-10-25

3. Teaching practices for English as foreign language at the tertiary level: A comprehensibility perspective;Frontiers in Education;2023-01-09

4. The Contribution of Phonological and Fluency Factors to Chinese L2 Comprehensibility Ratings: A Case Study of Urdu-speaking Learners;2022 13th International Symposium on Chinese Spoken Language Processing (ISCSLP);2022-12-11

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3