Development and validation of a neural network-based survival model for mortality prediction in ischemic heart disease

Abstract

Background The reduced precision of currently applied risk prediction models for patients with ischemic heart disease (IHD) is a limitation for clinical use. Using machine learning to integrate a much broader panel of features from electronic health records (EHRs) may improve precision markedly. Methods The development and validation of a prediction model for IHD in this study was based on Danish and Icelandic data from clinical quality databases, national registries, and electronic health records. Danish patients suspected for IHD and referred for a coronary angiography showing 1, 2, or 3 vessel-disease or diffuse coronary artery disease between 2006 and 2016 were included for model development (n = 39,746). Time to all-cause mortality, the prediction target, was tracked until 2019, or up to 5 years, whichever came first. To model time-to-event data and address censoring, neural network-based discrete-time survival models were used. Our prediction model, PMHnet, used up to 584 different features including clinical characteristics, laboratory findings, and diagnosis and procedure codes. Model performance was evaluated using time-dependent AUC (tdAUC) and the Brier score and was benchmarked against the updated GRACE risk score and less feature-rich neural network models. Models were evaluated using hold-out data (n = 5,000) and external validation data from Iceland (n = 8,287). Feature importance and model explainability factors were assessed using SHAP analysis. Findings : On the test set (n = 5,000), the tdAUC of PMHnet was 0.88[0.86–0.90] (case count = 196) at six months, 0.88[0.86–0.90] (cc = 261) at one year, 0.84[0.82–0.86] (cc = 395) at three years, and 0.82[0.80–0.84] (cc = 763) at five years. The model predictions were well-calibrated. PMHnet showed similar performance in the Icelandic data. Compared to the GRACE score and intermediate models limited to GRACE features or single data modalities, PMHnet had significantly better model discrimination across all evaluated prediction timepoints. Interpretation: More complex and feature-rich machine learning models improved prediction of all-cause mortality in patients with IHD and may be used to inform and guide clinical management.