Development and External Validation of a Machine Learning Model to Predict Pathological Complete Response after Neoadjuvant Chemotherapy in Breast Cancer: model development using commonly available clinical and demographic variables

Abstract

Abstract Purpose Several predictive models have been developed to predict pathological complete response (pCR) after neoadjuvant chemotherapy (NAC), but few of them are broadly applicable due to radiologic complexity and institution-specific clinical variables, and none have been externally validated. The purpose of this study was to develop and externally validate a machine learning model that predicts pCR following NAC in breast cancer patients using routinely collected clinical and demographic variables. Methods Electronic medical record data of patients with advanced breast cancer who received NAC prior to surgical resection from January 2017 to December 2020 were reviewed. Patient data from Hospital A was split into training and internal validation cohort. Five machine learning techniques including gradient boosting machine, support vector machine, random forest, decision tree and neural network were used to build predictive models and area under the receiver-operating characteristic curve (AUC) were compared to select the best model. Finally, the model was further validated in an independent cohort from Hospital B. Results A total of 1003 patients were included in the study: 287 in the training cohort, 71 in the internal validation cohort, and 645 in the external validation cohort. Overall, 36.3% of patients achieved pCR. Among the five machine learning models, gradient boosting machine showed the highest AUC for pCR prediction (AUC 0.903, 95% CI 0.833–0.972). External validation confirmed AUC of 0.833 (95% CI 0.800-0.865). Conclusion We used commonly available clinical and demographic variables to develop a machine learning model to predict pCR following NAC. External validation of the model demonstrated good discrimination power, which showed that routinely collected variables are sufficient to build a good prediction model.