Affiliation:
1. School of Foreign Languages, Xinyang Agriculture and Forestry University, Xinyang, Henan 464000, China
2. Office of International Exchange & Cooperation, Xinyang Agriculture and Forestry University, Xinyang, Henan 464000, China
Abstract
In recent years, economic globalization is the trend, and communication between countries is getting closer and closer; more and more people begin to pay attention to learning spoken English. The development of computer-aided language learning makes it more convenient for people to learn spoken English; however, the detection and correction of incorrect English pronunciation, which is its core, are still inadequate. In this paper, we propose a multimodal end-to-end English pronunciation error detection and correction model based on audio and video, which does not require phoneme forced alignment of the English pronunciation video signal to be processed, and uses rich audio and video features for English pronunciation error detection, which improves the error detection accuracy to a great extent especially in noisy environments. To address the shortcomings of the current lip feature extraction algorithm which is too complicated and not enough characterization ability, a feature extraction scheme based on the lip opening and closing angle is proposed. The lip syllable frames are obtained by video frame splitting, the syllables are denoised, the key point information of the lips is obtained using a gradient enhancement-based regression tree algorithm, the effects of speaker tilt and movement are removed by scale normalization, and finally, the lip opening and closing angles are calculated using mathematical geometry, and the lip feature values are generated by combining the angle changes.
Subject
Electrical and Electronic Engineering,Instrumentation,Control and Systems Engineering
Reference21 articles.
1. POLEMAD–A database for the multimodal analysis of Polish pronunciation
2. A study on the impact of Lombard effect on recognition of Hindi syllabic units using CNN based multimodal ASR systems;S. U. Maheswari;Archives of Acoustics,2020
3. LVID: a multimodal biometrics authentication system on smartphones;L. Wu;IEEE Transactions on Information Forensics and Security,2020
4. Multimodal wearable sensors to measure gait and voice;D. Psaltos;Digital biomarkers,2020
5. Sensor-based continuous authentication of smartphones’ users using behavioral biometrics: a contemporary survey;M. Abuhamad;IEEE Internet of Things Journal,2021
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献