BACKGROUND
Antiretroviral therapy (ART) has transformed HIV from a fatal illness to a chronic disease. Given the high rate of treatment interruptions, HIV programs employ a range of approaches to support individuals in adhering to ART and in reengaging those who interrupt treatment. These interventions can often be time consuming and costly, and thus providing for all may not be sustainable.
OBJECTIVE
To describe our experiences developing a machine learning (ML) model to predict interruption in treatment (IIT) at 30 days among people living with HIV (PLHIV) newly enrolled on ART in Nigeria and our integrating the model into the routine information system. To ascertain health worker perceptions and use of the model’s outputs for case management.
METHODS
Routine program data collected from January 2005 through February 2021 was used to train and test an ML model (boosting tree and extreme gradient boosting) to predict future IIT. Data were randomly sampled using an 80/20 split into a training and test data sets respectively. Model performance was estimated using sensitivity, specificity, and positive and negative predictive values. Variables considered to be highly associated with treatment interruption were pre-selected by a group of HIV prevention researchers, program experts, and biostatisticians for inclusion in the model. Individuals were defined as having IIT if they were provided a 30-day supply of antiretrovirals (ARVs) but did not return for a refill within 28 days of their scheduled follow-up visit date. Outputs from the ML model were shared weekly with health care workers at selected facilities.
RESULTS
After data cleaning, complete data for 136,747 clients were used for the analysis. The percentage of IIT cases decreased from 58.6% before 2017 to 14.2% during October 2019 through February 2021. Overall IIT was higher among clients who were sicker at enrollment. Other factors that were significantly associated with IIT included pregnancy, and breastfeeding status and facility characteristics (location, service level, and service type). Several models were initially developed; the selected model had sensitivity of 81%, specificity of 88%, PPV of 83% and NPV of 87%. and was successfully integrated into the national electronic medical records database. During field testing, the majority of users reported that an IIT prediction tool could lead to proactive steps for preventing IIT and improved patient outcomes.
CONCLUSIONS
High performing ML models to identify HIV patients at risk of IIT can be developed using routinely collected service delivery data and integrated into routine HMIS. Machine learning can improve the targeting of interventions through differentiated models of care before patients interrupt treatment resulting in increased cost effectiveness and improved patient outcomes.
CLINICALTRIAL