Evaluating Online Consumer Medication Information Systems: Comparative Online Usability Study (Preprint)

Abstract

Medication is the most common intervention in health care, and the number of online consumer information systems within the pharmaceutical sector is increasing. However, online consumer information systems can be a barrier for users, imposing information asymmetries between stakeholders.

The objective of this study was to quantify and compare the usability of an online consumer medication information system (OCMIS) against a reference implementation based on an interoperable information model for patients, physicians, and pharmacists.

Quantitative and qualitative data were acquired from patients, physicians, and pharmacists in this online usability study. We administered 3 use cases and a post hoc questionnaire per user. Quantitative usability data including effectiveness (task success), efficiency (task time), and user satisfaction (system usability scale [SUS]) was complemented by qualitative and demographic data. Users evaluated 6 existing systems and 1 reference implementation of an OCMIS.

A total of 137 patients, 81 physicians, and 68 pharmacists participated in this study. Task success varied from 84% to 92% in patients, 66% to 100% in physicians, and 50% to 91% in pharmacists. Task completion time decreased over the course of the study for all but 2 OCMIS within the patient group. Due to an assumed nonnormal distribution of SUS scores, within-group comparison was done using the Kruskal-Wallis test. Patients showed differences in SUS scores (<i>P</i>=.02) and task time (<i>P</i>=.03), while physicians did not have significant differences in SUS scores (<i>P</i>=.83) and task time (<i>P</i>=.72). For pharmacists, a significant difference in SUS scores (<i>P</i>&lt;.001) and task time (<i>P</i>=.007) was detected.

The vendor-neutral reference implementation based on an interoperable information model was proven to be a promising approach that was not inferior to existing solutions for patients and physicians. For pharmacists, it exceeded user satisfaction scores compared to other OCMIS. This data-driven approach based on an interoperable information model enables the development of more user-tailored features to increase usability. This fosters data democratization and empowers stakeholders within the pharmaceutical sector.