Leveraging Dual Usability Methods to Evaluate Clinical Decision Support among Traumatic Brain Injury Patients (Preprint)

Abstract

Traumatic brain injury patients (TBI) are at increased risk of developing venous thromboembolism (VTE). Clinical decision support systems (CDSSs) may improve the utilization of VTE prophylaxis protocols yet suffer from poor compliance among end-users due to a lack of user-centered design.

The objective of this research work was to improve the content, design, and workflow integration of a TBI-CDSS based on feedback from the experts and end-users.

The CDSS was evaluated leveraging a dual usability approach. A set of usability experts (N=3) and trauma providers (N=5) performed the heuristic evaluation (HE) and end-user testing (E-UT). Data was collected through triangulation of methods and analyzed using qualitative (thematic) and quantitative (descriptive) analyses.

We identified 145 total issues across both methods with 66 being unique i.e., 17 issues found by HE, 43 by E-UT, and 6 common ones. Thematic analysis was conducted on the unique issues (66) and was assigned to themes and subsequent sub-themes. We identified 13 unique themes. The three most identified themes were lack of supporting evidence (17 issues, representing 26% of the 66 issues), operational barriers arising from the test environment (11,17%), formatting inconsistencies, and lack of following standards (8, 12 %). The system's usability scale survey (SUS) score was 77.5 (std dev ±16) interpreted as an acceptable/good usability range.

Combining expert and end-user-driven usability evaluation methods led to the identification of a more comprehensive list of issues. This can facilitate the optimization of the TBI-CDSS, resulting in improved usability and care management.

This component of the project is not a clinical trial. The study protocol was submitted to the University of Minnesota Institutional Review Board and given the determination of “Exempt” as secondary research for which consent is not required. The mixed methods investigation was given the determination of “Not Human Research” as a quality improvement activity.