Usability Testing of a Web-Based Empathy Training Portal: Mixed Methods Study

Abstract

Background The prepandemic period saw a rise in web-based teaching. However, web-based tools for teaching the essential clinical skill of cognitive empathy (also known as perspective taking) remain limited. More of these tools are needed and require testing for ease of use and understanding by students. Objective This study aimed to evaluate the usability of the In Your Shoes web-based empathy training portal application for students using quantitative and qualitative methods. Methods This 3-phase formative usability study used a mixed methods design. In mid-2021, we conducted a remote observation of student participants interacting with our portal application. Their qualitative reflections were captured, followed by data analysis and iterative design refinements of the application. Overall, 8 third- and fourth-year nursing students from an undergraduate baccalaureate program at a Canadian university, in the western province of Manitoba, were included in this study. Participants in phases 1 and 2 were remotely observed by 3 research personnel while engaged in predefined tasks. In phase 3, two student participants were asked to use the application as they liked in their own environments, after which a video-recorded exit interview with a think-aloud process was conducted as participants responded to the System Usability Scale. We calculated descriptive statistics and performed content analysis to analyze the results. Results This small study included 8 students with a range of technology skills. Usability themes were based on participants’ comments on the application’s appearance, content, navigation, and functionality. The biggest issues that participants experienced were with navigating the application’s “tagging” features during video analysis and the length of educational material. We also observed variations in 2 participants’ system usability scores in phase 3. This may be because of their different comfort levels with technology; however, additional research is required. We made iterative refinements to our prototype application (eg, added pop-up messages and provided a narrated video on the application’s “tagging” function) based on participant feedback. Conclusions With increasing engagement in web-based teaching, technology has become an essential medium for receiving health care education. We developed a novel prototype application as a supplemental classroom tool to foster students’ self-directed learning of empathy. This study provided direction for refinements to optimize the usability of and satisfaction with this innovative application. Qualitative feedback revealed favorable input toward learning perspective taking place on the web and helpful recommendations for improving user experiences with the application. We could not fully assess the application’s key functions owing to the COVID-19 protocols. Thus, our next step is to obtain feedback from a larger sample of student users, whose experiences performing “live” video capture, annotation, and analysis will be more authentic and wholesome with the refined application. We discuss our findings in relation to research on nursing education, perspective taking, and adaptive e-learning.