Barriers and Enablers which Affect the Implementation of Intelligent Guidance Systems for Patients in Chinese Tertiary Transfer Hospitals: Usability Evaluation (Preprint)

Abstract

Hospital information systems have been widely adopted across China, in recent years, significantly changing the way in which hospitals manage their processes and patient visits. Intelligent Guidance Systems for Patients (IGSP), which resemble humanoid characteristics using Artificial Intelligence (AI), assist patients in navigating hospital visits, obtaining medical guidance, consultation, and other medical services. Despite their widespread adoption in tertiary transfer hospitals in China, usability studies on such systems are scarce. To date, there is no practical or standardized measurement of system usability, leading to difficult inspection, maintenance and servicing processes.

IGSP can improve user experiences pre, during and post hospital visits. This study aims to determine the usability deficiency of IGSP and understand how various factors influence users’ satisfaction during use.

Employing the requirements set out in the ISO9214-11:2018 standard, two inspection methods were used with 3 experts and 360 recruited end-users. First, the Heuristic evaluation method was employed to detect usability problems and to demonstrate any violations of Nielsen’s 10 heuristic principles. Second, the System Usability Scale (SUS) was applied to evaluate participants’ satisfaction towards IGSP. Finally, analysis of variance tests and a multiple linear regression analysis was performed to establish correlations between user satisfaction and various characteristics.

A total of 78 problems violated the heuristic principles for 169 times. These problems were divided into five categories: voice interaction, in-hospital navigation, medical consultation, interactive interface design, and miscellaneous. The average SUS score for IGSP was 72.8 (SD 14.69), indicating a good usability level. User satisfaction scores differed among hospitals (F=3.513, P=.031), age of user (F=6.010, P=.000), and smartphone usage time (F=4.781, P=.000). The multiple linear regression model reflected participants’ education level (t=-2.41, p= .016) and gender (t=-2.080, p=.038), while medical industry (t=-2.074, p=0.039) had a negative effect on user satisfaction scores. Conversely, daily smartphone usage times (t= 2.470, p=.014) helped to increase user satisfaction scores.

This study demonstrates that IGSP can improve user experiences during interaction between patient and hospital. However, system usability deficiencies did exist; some of which must be addressed. Our findings are considered conducive to tertiary transfer hospitals in developing user-friendly IGSP for improving patient experiences. Tertiary hospitals should take full advantage of IGSP to help reduce waiting times and meet the various needs of patients pre, during and post hospital visit. User-centered conception should be integrated into the process of design and modification of IGSP.