Integrating Large Language Models (LLMs) and Deep Representations of Emotional Features for the Recognition and Evaluation of Emotions in Spoken English

Abstract

This study is dedicated to developing an innovative method for evaluating spoken English by integrating large language models (LLMs) with effective space learning, focusing on the analysis and evaluation of emotional features in spoken language. Addressing the limitation of current spoken English evaluation software that primarily focuses on acoustic features of speech (such as pronunciation, frequency, and prosody) while neglecting emotional expression, this paper proposes a method capable of deeply recognizing and evaluating emotional features in speech. The core of the method comprises three main parts: (1) the creation of a comprehensive spoken English emotion evaluation dataset combining emotionally rich speech data synthesized using LLMs with the IEMOCAP dataset and student spoken audio; (2) an emotion feature encoding network based on transformer architecture, dedicated to extracting effective spatial features from audio; (3) an emotion evaluation network for the spoken English language that accurately identifies emotions expressed by Chinese students by analyzing different audio characteristics. By decoupling emotional features from other sound characteristics in spoken English, this study achieves automated emotional evaluation. This method not only provides Chinese students with the opportunity to improve their ability to express emotions in spoken English but also opens new research directions in the fields of spoken English teaching and emotional expression evaluation.