Effectiveness of a Vendor Predictive Model for the Risk of Pediatric Asthma Exacerbation: A Difference-in-Differences Analysis

Abstract

Abstract Background Asthma is a common cause of morbidity and mortality in children. Predictive models may help providers tailor asthma therapies to an individual's exacerbation risk. The effectiveness of asthma risk scores on provider behavior and pediatric asthma outcomes remains unknown. Objective Determine the impact of an electronic health record (EHR) vendor-released model on outcomes for children with asthma. Methods The Epic Systems Risk of Pediatric Asthma Exacerbation model was implemented on February 24, 2021, for volunteer pediatric allergy and pulmonology providers as a noninterruptive risk score visible in the patient schedule view. Asthma hospitalizations, emergency department (ED) visits, or oral steroid courses within 90 days of the index visit were compared from February 24, 2019, to February 23, 2022, using a difference-in-differences design with a control group of visits to providers in the same departments. Volunteer providers were interviewed to identify barriers and facilitators to model use. Results In the intervention group, asthma hospitalizations within 90 days decreased from 1.4% (54/3,842) to 0.7% (14/2,165) after implementation with no significant change in the control group (0.9% [171/19,865] preimplementation to 1.0% [105/10,743] post). ED visits in the intervention group decreased from 5.8% (222/3,842) to 5.5% (118/2,164) but increased from 5.5% (1,099/19,865) to 6.8% (727/10,743) in the control group. The adjusted difference-in-differences estimators for hospitalization, ED visit, and oral steroid outcomes were −0.9% (95% confidence interval [CI]: −1.6 to −0.3), –2.4% (−3.9 to −0.8), and –1.9% (−4.3 to 0.5). In qualitative analysis, providers understood the purpose of the model and felt it was useful to flag high exacerbation risk. Trust in the model was calibrated against providers' own clinical judgement. Conclusion This EHR vendor model implementation was associated with a significant decrease in asthma hospitalization and ED visits within 90 days of pediatric allergy and pulmonology clinic visits, but not oral steroid courses.