Exploring usage-based and usability metrics for user experience for sustainable e-learning systems

Abstract

The digital revolution has ushered in a new era of education, characterized by the widespread adoption of e-learning systems. While these systems offer unprecedented opportunities for knowledge dissemination, the challenge lies in ensuring an effective and sustainable user experience. Although many studies have investigated the usability of e-learning systems, limited works have addressed a combination of usage-based metrics and Usability Metrics for User Experience (UMUX). This research article explores the intersection of usage-based and usability metrics within sustainable e-learning systems, aiming to develop a comprehensive framework for assessment and improvement. The study was conducted in a vocational school and participated by 64 teachers. They were invited to complete four tasks in an e-learning system and asked to fill out the questionnaires afterward. Besides calculating UMUX and UMUX-Lite scores, three usage-based metrics were also measured: completion rate, task duration, and the number of mouse clicks. The results showed that UMUX and UMUX-Lite had the potential to replace the System Usability Scale (SUS), which contains ten-item questionnaires and has been the industry standard over time. Shorter questionnaires, like UMUX, which includes four items, and UMUX-Lite, which even has only two items, would be preferred when the usability study has a limited time. In addition, the UMUX and UMUX-Lite scores correlated highly with the selected usage-based metrics, like the SUS score.