The effects of different multimodal expressions on English learners of varying proficiency levels

Abstract

Abstract Texts containing information in different modes are growing in number, technological sophistication, and quality presentation. In this changing semiotic landscape, it is timely to examine which mode combination(s) is most comprehensible for English learners. Accordingly, three text conditions – a verbal text, a supplementary text, and a complementary text – were examined for which mode combination(s) produces the best comprehension outcomes for English learners. Participants were intermediate learners (n = 111) and advanced learners (n = 111). Each language group was further divided into three subgroups (n = 37), and each of the subgroups was tasked with reading in one text condition and answering questions about the text. Participants’ comprehension scores were compared across the text conditions and language groups. The results revealed that the complementary text condition significantly impedes learners’ comprehension, and no significant difference was observed between the verbal text and the supplementary text conditions. These results indicate that learners did not engage in multimodal reading by integrating the various modes at interplay in the texts. For a complete literacy competence, these findings raise the need for engaging learners in deciphering texts from a multimodal perspective by establishing connections across the semiotic resources present in a text.