A Framework of the Training Module for Untrained Observers in Usability Evaluation Motivated by COVID-19: Enhancing the Validity of Usability Evaluation for Children’s Educational Games

Abstract

The usability evaluation of educational games is an important task, especially for children. By applying Jakob Nielsen’s ten heuristics, most of the HCI designs can be evaluated, but when educational games are involved, where the user being observed is a child between the ages of six and eight, many questions arise. Is the observer trained well enough to observe the child’s reactions to the game with regard to its memorability, learnability, ease of use, and enjoyment? Will it be necessary for the observer to have a training session exploring the game before evaluating a child? Our research suggests that a training module designed to train an untrained facilitator (observer) in how to evaluate four usability dimensions (learnability, memorability, ease of use, and enjoyment) would be very useful. The usability evaluation data was collected by observing users playing generic educational games, using the Mann–Whitney U test, which was conducted by two groups of observers, one trained and one untrained. This was then reviewed, and a distinct difference was found between the results of evaluations in the two groups, thus validating the importance of training for an observer.