Explainable Machine Learning model reveals its decision-making process in identifying patients with paroxysmal atrial fibrillation at high risk for recurrence after catheter ablation

Abstract

Abstract Background A number of models have been reported for predicting atrial fibrillation (AF) recurrence after catheter ablation. Although many machine learning (ML) models were developed among them, black-box effect existed widely. It was always difficult to explain how variables affect model output. We sought to implement an explainable ML model and then reveal its decision-making process in identifying patients with paroxysmal AF at high risk for recurrence after catheter ablation. Methods Between January 2018 and December 2020, 471 consecutive patients with paroxysmal AF who had their first catheter ablation procedure were retrospectively enrolled. Patients were randomly assigned into training cohort (70%) and testing cohort (30%). The explainable ML model based on Random Forest (RF) algorithm was developed and modified on training cohort, and tested on testing cohort. In order to gain insight into the association between observed values and model output, Shapley additive explanations (SHAP) analysis was used to visualize the ML model. Results In this cohort, 135 patients (14.2/100 patient-years) experienced tachycardias recurrence. With hyperparameters adjusted, the ML model predicted AF recurrence with an area under the curve (AUC) of 66.7% in the testing cohort. Based on SHAP analysis, the ML model's decision-making process was revealed: (i) summary plot listed the top 15 features in descending order and preliminary showed the association between features and outcome prediction; (ii) dependence plots combined with force plots showed the impact of single feature on model output, and helped determine high risk cut-off points; (iii) decision plot recognized significant outliers. Conclusion An explainable ML model effectively revealed its decision-making process in identifying patients with paroxysmal atrial fibrillation at high risk for recurrence after catheter ablation. Physicians can combine model output, visualization of model and clinical experience to make better decision.