Usability and Clinician Acceptance of a Deep Learning-Based Clinical Decision Support Tool for Predicting Glaucomatous Visual Field Progression

Abstract

Précis: We updated a clinical decision support tool integrating predicted visual field (VF) metrics from an artificial intelligence model and assessed clinician perceptions of the predicted VF metric in this usability study. Purpose: To evaluate clinician perceptions of a prototyped clinical decision support (CDS) tool that integrates visual field (VF) metric predictions from artificial intelligence (AI) models. Methods: Ten ophthalmologists and optometrists from the University of California San Diego participated in 6 cases from 6 patients, consisting of 11 eyes, uploaded to a CDS tool (“GLANCE”, designed to help clinicians “at a glance”). For each case, clinicians answered questions about management recommendations and attitudes towards GLANCE, particularly regarding the utility and trustworthiness of the AI-predicted VF metrics and willingness to decrease VF testing frequency. Main Outcome(s) and Measure(s): Mean counts of management recommendations and mean Likert scale scores were calculated to assess overall management trends and attitudes towards the CDS tool for each case. In addition, system usability scale scores were calculated. Results: The mean Likert scores for trust in and utility of the predicted VF metric and clinician willingness to decrease VF testing frequency were 3.27, 3.42, and 2.64, respectively (1=strongly disagree, 5=strongly agree). When stratified by glaucoma severity, all mean Likert scores decreased as severity increased. The system usability scale score across all responders was 66.1±16.0 (43rd percentile). Conclusions: A CDS tool can be designed to present AI model outputs in a useful, trustworthy manner that clinicians are generally willing to integrate into their clinical decision-making. Future work is needed to understand how to best develop explainable and trustworthy CDS tools integrating AI before clinical deployment.