Explainable AI toward understanding the performance of the top three TADPOLE Challenge methods in the forecast of Alzheimer’s disease diagnosis

Abstract

The Alzheimer′s Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge is the most comprehensive challenge to date with regard to the number of subjects, considered features, and challenge participants. The initial objective of TADPOLE was the identification of the most predictive data, features, and methods for the progression of subjects at risk of developing Alzheimer′s. The challenge was successful in recognizing tree-based ensemble methods such as gradient boosting and random forest as the best methods for the prognosis of the clinical status in Alzheimer’s disease (AD). However, the challenge outcome was limited to which combination of data processing and methods exhibits the best accuracy; hence, it is difficult to determine the contribution of the methods to the accuracy. The quantification of feature importance was globally approached by all the challenge participant methods. In addition, TADPOLE provided general answers that focused on improving performance while ignoring important issues such as interpretability. The purpose of this study is to intensively explore the models of the top three TADPOLE Challenge methods in a common framework for fair comparison. In addition, for these models, the most meaningful features for the prognosis of the clinical status of AD are studied and the contribution of each feature to the accuracy of the methods is quantified. We provide plausible explanations as to why the methods achieve such accuracy, and we investigate whether the methods use information coherent with clinical knowledge. Finally, we approach these issues through the analysis of SHapley Additive exPlanations (SHAP) values, a technique that has recently attracted increasing attention in the field of explainable artificial intelligence (XAI).