Assessing How Clinicians and Staff Would Use a Diabetes Artificial Intelligence Prediction Tool (Preprint)

Abstract

Nearly one third of those with diabetes are poorly controlled (hemoglobin A1c of ≥9.0%). Identifying at-risk individuals and providing them with effective treatment is an important strategy for preventing poor control.

The objective is to assess how clinicians and staff would use a clinical decision support tool based on artificial intelligence (AI) and identify factors that affect adoption.

This is a mixed-methods study that combines semi-structured interviews and surveys to assess the perceived usefulness and ease of use, intent to use, and factors affecting tool adoption. We recruited clinicians and staff from practices that manage diabetes. During interviews, participants reviewed a sample electronic health record alert and were informed that the tool uses AI to identify those at high risk for poor control. Participants discussed how they would use the tool, whether it would contribute to care, and the factors affecting its implementation. In a survey, participants reported their demographics, rank ordered factors influencing adoption of the tool, and their perception of the tool’s usefulness as well as their intent to use, ease of use, and organizational support for use. Qualitative data were analyzed using a thematic content analysis approach. We used descriptive statistics to report demographics and analyze the findings of the survey.

Twenty-two individuals participated in the study. Two-thirds of respondents were physicians. Thirty-six percent worked in academic health centers while 27% worked in Federally Qualified Health Centers. The interviews identified several themes: (1) This tool has the potential to be useful because it provides information that is not currently available and can make care more efficient and effective; (2) Clinicians and staff were concerned about how the tool affects patient-oriented outcomes and clinic workflows; (3) Adoption of the tool is dependent on its validation, transparency, actionability, and design and could be increased with changes to the interface and usability; and (4) Implementation would require buy in and need to be tailored to the demands and resources of clinics and communities. Survey findings supported these themes as seventeen (77.3%) participants somewhat, moderately, or strongly agreed that they would use the tool while these figures were 18 (81.8%) for usefulness, 18 (81.8%) for ease-of-use, and 15 (68.2%) for clinic support, respectively. The two highest-ranked factors affecting adoption were whether the tool improves health and the accuracy of the tool.

A majority found the tool to be easy to use and useful though they had concerns about alert fatigue, bias, and transparency. These data will be used to enhance the design of an AI tool.