Human-Centered Design with Usability Evaluation of a Remote Patient Monitoring System for Intensive Care Medicine: A Mixed-Methods Study (Preprint)

Abstract

Continuous monitoring of vital signs is critical for ensuring patient safety in intensive care units (ICUs) and is becoming increasingly relevant in general wards. The effectiveness of health information technologies such as patient monitoring systems is highly determined by usability, the lack of which can ultimately compromise patient safety. Usability problems can be identified and prevented by involving users (ie, clinicians).

In this study, we applied a human-centered design (HCD) approach to evaluate the usability of a remote patient monitoring system user interface (UI) in the ICU context, and to conceptualize and evaluate design changes.

Following IRB approval (EA1/031/18), a formative evaluation of the monitoring UI was performed. Simulated use tests with think-aloud protocols were conducted with ICU staff (n = 5) and the resulting qualitative data was analyzed using a deductive analytic approach. Based on the identified usability problems, we conceptualized informed design changes and applied them to develop an improved prototype of the monitoring UI. Comparing the UIs, we evaluated perceived usability with the system usability scale (SUS), performance efficiency with the normative path deviation (NPD), and effectiveness by measuring the task completion rate. Measures were tested for statistical significance using a two-sample t test, Poisson regression with a generalized linear mixed-effects model, and the N-1 chi-square test. P values <.05 were considered significant.

We found 37 individual usability problems specific to the monitoring UI, which could be assigned to 6 subcodes: Usefulness of the system, response time, responsiveness, meaning of labels, function of UI elements, and navigation. Among user ideas and requirements for the UI were high usability, customizability, and the provision of audible alarm notifications. Changes in graphics and design were proposed to allow better navigation, information retrieval, and spatial orientation. The UI was revised by creating a prototype with a more responsive design and changes regarding labeling and UI elements. Statistical analysis showed that perceived usability improved significantly (SUS Design A: mean 68.5, SD 11.26, n = 5; Design B: mean 89, SD 4.87, n = 5; P = .003), as did performance efficiency (NPD Design A: mean 8.8, SD 5. 26, n = 5; Design B: mean 3.2, SD 3.03, n = 5; P = .001), and effectiveness (Design A: 18 trials, failed 7 times, passed 11 times; Design B: 20 trials, failed 0 times, passed 20 times; P = .002).

Usability testing with think-aloud protocols led to a patient monitoring UI with a significantly improved usability, performance and effectiveness. In the ICU work environment, difficult-to-use technology may result in detrimental outcomes for staff and patients. Technical devices should be designed to support efficient and effective work processes. Our results suggest that this can be achieved by applying basic HCD methods and principles.

ClinicalTrials.gov NCT03514173