Formative Usability Evaluation of a Three-Way Digital Healthcare System for the People with Disabilities and Their Caregivers: A Cross-Sectional Study

Abstract

During the COVID-19 pandemic, there was a growing awareness about the importance of building a health and safety net based on digital healthcare systems, such as ICT-based local community online services and patient monitoring technology. This study was conducted with the aim of evaluating the formative usability of a three-way digital healthcare system, which had been developed to build a health and safety net for people with disabilities and deriving the directions for system improvement in order for them to be used as basic data for further system enhancement. A formative usability evaluation of a three-way digital healthcare system was performed with the participation of 43 healthcare professionals, using the 10-item System Usability Scale (SUS) and five items for satisfaction evaluation. Each item was rated on a five-point Likert scale, with the result converted to a scale of 100. Analysis was performed using the average score and the acceptable system usability level. The overall mean SUS score was 62.4, which corresponds to Grade D according to the SUS grading scale, and the below-average items were complexity (Q2), convenience (Q8), simplicity (Q3), professionalism (technician support, prior learning) (Q4, Q10), and learnability (Q7). The overall mean user satisfaction was 71.2 points, where overall satisfaction, system architecture and understandability, and continuous use intention were marked with below-average scores. The SUS D grade is interpreted as “fair” and the water solubility is “almost acceptable”. For the usability enhancement of the newly developed a three-way digital healthcare system, the overall direction for system architecture improvement was analyzed centering on complexity (Q2), convenience (Q8), professionalism (technician support, prior learning) (Q4, Q10), learnability (Q7), and simplicity (Q3). Efforts need to be directed at enhancing system satisfaction and continuance rate by deriving detailed system improvement strategies and achieving system enhancement to reflect the opinions of not only experts but also users.