Are scale-based techniques enough for learners to convey their UX when using a Learning Management System?

Abstract

As technology evolves, learning institutions have sought to invest in new learning technologies. Many universities have been adopting platforms designed to support the learning process, called Learning Management Systems (LMSs), to complement face-to-face learning, as well as to support distance learning. This widespread adoption of LMSs raises the need for evaluating them regarding aspects related to quality in use, such as User eXperience (UX). UX is very important in e-learning context, given that it may influence learner’s attitude towards the use of the LMS. Despite its importance, few studies evaluated the UX of LMSs. Moreover, researchers have been using generic techniques to evaluate them. In this context, there is no evidence of studies conducted to identify whether these techniques can fully capture the experience conveyed by these platforms. This paper presents an analysis of two UX evaluation techniques (User Experience Questionnaire, and Integrated Experience Acceptance Model) applied to an LMS called Edmodo. We carried out a study with 34 students, divided into two groups, to evaluate the UX of Edmodo and obtain their perceptions about the technique they used. We also gathered the difficulties faced by the students during the execution of the tasks on the platform, while correlating those difficulties with learning issues. In addition, we performed a qualitative analysis in order to better understand the results. Regarding Edmodo, the results showed that the students perceived its UX as positive. Regarding the techniques, both groups considered them easy to use and useful. However, around 41% of the students in both groups reported that they were not able to fully evaluate their UX using the techniques. The qualitative results allowed the identification of positive and negative aspects that researchers may consider during the improvement or development of new UX evaluation techniques.