No-show Prediction Model Performance Among People With HIV: External Validation Study

Abstract

Background Regular medical care is important for people living with HIV. A no-show predictive model among people with HIV could improve clinical care by allowing providers to proactively engage patients at high risk of missing appointments. Epic, a major provider of electronic medical record systems, created a model that predicts a patient’s probability of being a no-show for an outpatient health care appointment; however, this model has not been externally validated in people with HIV. Objective We examined the performance of Epic’s no-show model among people with HIV at an academic medical center and assessed whether the performance was impacted by the addition of demographic and HIV clinical information. Methods We obtained encounter data from all in-person appointments among people with HIV from January 21 to March 30, 2022, at the University of Chicago Medicine. We compared the predicted no-show probability at the time of the encounter to the actual outcome of these appointments. We also examined the performance of the Epic model among people with HIV for only HIV care appointments in the infectious diseases department. We further compared the no-show model among people with HIV for HIV care appointments to an alternate random forest model we created using a subset of seven readily accessible features used in the Epic model and four additional features related to HIV clinical care or demographics. Results We identified 674 people with HIV who contributed 1406 total scheduled in-person appointments during the study period. Of those, we identified 331 people with HIV who contributed 440 HIV care appointments. The performance of the Epic model among people with HIV for all appointments in any outpatient clinic had an area under the receiver operating characteristic curve (AUC) of 0.65 (95% CI 0.63-0.66) and for only HIV care appointments had an AUC of 0.63 (95% CI 0.59-0.67). The alternate model we created for people with HIV attending HIV care appointments had an AUC of 0.78 (95% CI 0.75-0.82), a significant improvement over the Epic model restricted to HIV care appointments (P<.001). Features identified as important in the alternate model included lead time, appointment length, HIV viral load >200 copies per mL, lower CD4 T cell counts (both 50 to <200 cells/mm3 and 200 to <350 cells/mm3), and female sex. Conclusions For both models among people with HIV, performance was significantly lower than reported by Epic. The improvement in the performance of the alternate model over the proprietary Epic model demonstrates that, among people with HIV, the inclusion of demographic information may enhance the prediction of appointment attendance. The alternate model further reveals that the prediction of appointment attendance in people with HIV can be improved by using HIV clinical information such as CD4 count and HIV viral load test results as features in the model.