Improving accuracy of American Society of Anesthesiologists Physical Status using audit and feedback and artificial intelligence: a time-series analysis

Abstract

Abstract Background While the American Society of Anesthesiologists (ASA) Physical Status (PS) is used to adjust for greater mortality risk with higher ASA PS classification, inaccurate classification can lead to an inaccurate comparison of institutions. Objective The purpose of this study was to assess the effect of audit and feedback with a rule-based artificial intelligence algorithm on the accuracy of ASA PS classification. Methods We reviewed 78 121 anesthetic records from 1 January 2017 to 19 February 2020. The first intervention entailed audit and feedback emphasizing accurately documenting ASA PS classification using body mass index (BMI), while the second intervention consisted of implementing a rule-based artificial intelligence algorithm. If a patient with a BMI ≥40 kg/m2 had a documented ASA PS classification of 1 or 2, the provider was alerted to change the ASA PS classification to 3 or above. The primary outcome was the overall proportion of patients with inaccurate ASA PS classification based on BMI per month. Secondary outcomes included the proportion of patients with a BMI ≥40 or a BMI 30–39.9 who had inaccurate ASA PS classification and the proportion of patients documented as having ASA 3–5. Data were analyzed using interrupted time-series analysis. Results For the primary outcome, the slope for ASA PS classification inaccurately incorporating BMI was unchanging before the first intervention (parameter coefficient 0.002, 95% CI −0.034 to 0.038; P = 0.911). Following the first intervention, there was an immediate level change (parameter coefficient −0.821, 95% CI −1.236 to −0.0406; P < 0.001) without significant change in slope (parameter coefficient −0.048, 95% CI −0.100 to 0.004; P = 0.067). The post-intervention slope was negative (parameter coefficient −0.046, 95% CI −0.083 to −0.009; P = 0.017). Following the second intervention, there was no level change (parameter coefficient 0.203, 95% CI −0.380 to 0.463; P = 0.839) and no significant change in slope (parameter coefficient 0.013, 95% CI −0.043 to 0.043; P = 0.641). The post-intervention slope was not significant (parameter coefficient −0.034, 95% CI −0.078 to 0.010; P = 0.121). The proportion of patients whose ASA PS classification inaccurately incorporated BMI at the first and final timepoint of the study was 2.6% and 0.8%, respectively. Conclusions Our quality improvement efforts successfully modified clinician behavior to accurately incorporate BMI into the ASA PS classification. By combining audit and feedback methodology with a rule-based artificial intelligence algorithm, we created a process that resulted in immediate and sustained effects. Improving ASA PS classification accuracy is important because it affects quality metrics, research design, resource allocation and workflow processes.