Developing and validating a machine learning ensemble model to predict postoperative delirium in a cohort of high-risk surgical patients

Abstract

BACKGROUND Postoperative delirium (POD) has a negative impact on prognosis, length of stay and the burden of care. Although its prediction and identification may improve postoperative care, this need is largely unmet in the Brazilian public health system. OBJECTIVE To develop and validate a machine-learning prediction model and estimate the incidence of delirium. We hypothesised that an ensemble machine-learning prediction model that incorporates predisposing and precipitating features could accurately predict POD. DESIGN A secondary analysis nested in a cohort of high-risk surgical patients. SETTING An 800-bed, quaternary university-affiliated teaching hospital in Southern Brazil. We included patients operated on from September 2015 to February 2020. PATIENTS We recruited 1453 inpatients with an all-cause postoperative 30-day mortality risk greater than 5% assessed preoperatively by the ExCare Model. MAIN OUTCOME MEASURE The incidence of POD classified by the Confusion Assessment Method, up to 7 days postoperatively. Predictive model performance with different feature scenarios were compared with the area under the receiver operating characteristic curve. RESULTS The cumulative incidence of delirium was 117, giving an absolute risk of 8.05/100 patients. We developed multiple machine-learning nested cross-validated ensemble models. We selected features through partial dependence plot analysis and theoretical framework. We treated the class imbalance with undersampling. Different feature scenarios included: 52 preoperative, 60 postoperative and only three features (age, preoperative length of stay and the number of postoperative complications). The mean areas (95% confidence interval) under the curve ranged from 0.61 (0.59 to 0.63) to 0.74 (0.73 to 0.75). CONCLUSION A predictive model composed of three indicative readily available features performed better than those with numerous perioperative features, pointing to its feasibility as a prognostic tool for POD. Further research is required to test the generalisability of this model. TRIAL REGISTRATION Institutional Review Board Registration number 04448018.8.0000.5327 (Brazilian CEP/CONEP System, available in https://plataformabrasil.saude.gov.br/).