User Testing of a Diagnostic Decision Support System with Machine-assisted Chart Review to Facilitate Clinical Genomic Diagnosis

Abstract

ABSTRACTBackgroundThere is a need in clinical genomics for systems that assist in clinical diagnosis, analysis of genomic information and periodic re-analysis of results, and can utilize information from the electronic health record to do so. Such systems should be built using the concepts of human-centered design, fit within clinical workflows, and provide solutions to priority problems.MethodsWe adapted a commercially available diagnostic decision support system (DDSS) to use extracted findings from a patient record and combine them with genomic variant information in the DDSS interface. Three representative patient cases were created in a simulated clinical environment for user testing. A semi-structured interview guide was created to illuminate factors relevant to human factors in CDS design and organizational implementation.ResultsSix individuals completed the user testing process. Tester responses were positive and noted good fit with real-world clinical genetics workflow. Technical issues related to interface, interaction, and design were minor and fixable. Testers suggested solving issues related to terminology and usability through training and infobuttons. Time savings was estimated at 30-50% and additional uses such as in-house clinical variant analysis were suggested for increase fit with workflow and to further address priority problems.ConclusionThis study provides preliminary evidence for usability, workflow fit, acceptability, and implementation potential of a modified DDSS that includes machine-assisted chart review. Continued development and testing using principles from human-centered design and implementation science are necessary to improve technical functionality and acceptability for multiple stakeholders and organizational implementation potential to improve the genomic diagnosis process.SUMMARYWhat is already known?There is a need in clinical genomics for tools that assist in analysis of genomic information and can do so using information from the electronic health record.Such tools should be easy to use, fit within clinical workflows, and provide solutions to priority problems as defined by clinician end-users.Natural language processing (NLP) is a useful tool to read patient records and extract findings.What does this paper add?We demonstrated the use of Human-centered design and implementation science principles in a simulated environment for assessment of a new version of a decision support tool prior to large-scale implementation.This study provides preliminary evidence that a clinical decision support tool with machine-assisted chart review is acceptable to clinical end-users, fits within the clinical workflow, and addresses perceived needs within the differential diagnosis process across all Mendelian genetic disorders.Terminology codes for DDSSs should have levels of granularity tuned to the sensitivity and specificity appropriate to its various functions, e.g., NLP versus chart documentation.