Effects of virtual reality and test environment on user experience, usability, and mental workload in the evaluation of a blood pressure monitor

Abstract

User experience and user acceptance of a product are essential for the product’s success. Virtual reality (VR) technology has the potential to assess these parameters early in the development process. However, research is scarce on whether the evaluation of the user experience and user acceptance of prototypes in VR, as well as the simulation of the usage environment, lead to comparable results to reality. To investigate this, a digital twin of a blood pressure monitor (BPM) was created using VR. In a 2 × 2 factorial between-subjects design, 48 participants tested the real or VR BPM. The tests were performed either in a low-detail room at a desk or in a detailed operating room (OR) environment. Participants executed three use scenarios with the BPM and rated their user experience and acceptance with standardized questionnaires. A test leader evaluated the performance of the participants’ actions using a three-point scheme. The number of user interactions, task time, and perceived workload were assessed. The participants rated the user experience of the BPM significantly (p< .05) better in VR. User acceptance was significantly higher when the device was tested in VR and in a detailed OR environment. Participant performance and time on task did not significantly differ between VR and reality. However, there was significantly less interaction with the VR device (p< .001). Participants who tested the device in a detailed OR environment rated their performance significantly worse. In reality, the participants were able to haptically experience the device and thus better assess its quality. Overall, this study shows that user evaluations in VR should focus on objective criteria, such as user errors. Subjective criteria, such as user experience, are significantly biased by VR.