Speech Emotion Recognition in Native and Nonnative Languages

Abstract

Abstract Background: Speech emotion recognition can be experimentally tested in native and nonnative languages, as the mechanisms involved would differ across these languages. The recognition of speech in the native language is mediated through the language, and in nonnative languages, speech emotion recognition is facilitated through affective prosody. Linguistic prosody is anchored by the left hemisphere, whereas the right hemisphere coordinates the right hemisphere. The current study was carried out with the aim of testing speech emotion recognition in native and nonnative speakers with the motive of assessing the basic role of language in mediating prosody. Methods: The study was carried out on native and nonnative speakers of Malayalam, Assamese, and Bengali. Fifteen sentences (5 declarative, 5 interrogative, and 5 exclamatory) were recorded from native speakers of the aforementioned languages; these sentences were played to native speakers and nonnative speakers of a given language. For instance, the sentences in Malayalam were played to native speakers of Malayalam and nonnative speakers (speakers of Assamese and Bengali). The response sheet had three smileys, and the participants were asked to mark the appropriate smiley after listening to the sentences. Each correct response was given a score of 1, whereas an incorrect response was given a score of 0. Results: The native speakers of Malayalam, Assamese, and Bengali secured scores of 13.14 and 15 in their native languages, respectively. In the nonnative language, the participants secured scores in the range of 2–5. The Kruskal–Wallis test showed a significant difference between the native and nonnative languages. The results depicted that the native speakers of a language were able to identify the speech emotions easily, whereas they had difficulty identifying the sentences in the other two languages. The same trend was observed for speakers of Malayalam, Assamese, and Bengali. This showed that speech emotion recognition was linguistically driven. Conclusions: The results highlight that the prosody is more linguistically driven as the performance in the native language was better than in nonnative languages.