Development and Validation of Prediction Models for High Flow Nasal Cannula Failure in Patients with Acute Hypoxic Respiratory Failure: A Machine-Learning Approach (Preprint)

Abstract

Background: Acute hypoxic respiratory failure (AHRF) accounts for a large proportion of intensive care unit admissions. High flow nasal cannula (HFNC) is an emerging respiratory support technique that may improve oxygenation levels in patients. However, failure of HFNC may result in delayed intubation, prolonged mechanical ventilation and increased risk of increased mortality. Timely and accurate prediction of HFNC failure is of clinical importance.

Objective: to develope and validate prediction models for high flow nasal cannula failure in patients with acute hypoxic respiratory failure.

Methods: Firstly, the least absolute shrinkage and selection operator regression analysis was used as the feature selection for HFNC failure, and the features suitable for constructing the prediction model were selected by the lowest λ of the minimum mean cross-validation error. Next, four machine learning algorithms, C5.0, random forest (RF), extreme random tree (ERT) and extreme gradient boosting (XGBoost), were selected to construct the prediction models. Finally, the models are validated using receiver operating characteristic (ROC) curves and exact recall (PR) curves, model evaluation metrics, calibration plots and decision curve analysis, and further evaluated by internal validation.

Results: The study included 739 patients. Of those, 232 (31.4%) patients experienced HFNC failure. The areas under the receiver operating characteristic curves (AUROCs) of the C5.0, random forest, extremely randomized trees , and Extreme Gradient Boosting models were 0.807, 0.819, 0.816, and 0.818, respectively, which were higher than the ROX (AUROC=0.603) and mROX (AUROC=0.579) indexes. Similarly, the areas under the precision-recall curves (AUPRs) of the C5.0, RF, ERT, and XGBoost models were 0.813, 0.837, 0.832, and 0.839, respectively, which were higher than the ROX (AUPR=0.646) and mROX (AUPR=0.609) indexes. The ERT model had the most balanced predictive performance (sensitivity 0.701, specificity 0.817, accuracy 0.761, and balanced accuracy 0.759). The calibration curves demonstrated that the predicted risk probabilities among the ML models and the observed probabilities maintained good consistency. The results of the decision-curve analysis indicated the clinical validity of the ML model.

Conclusions: This study has developed an interpretable ML model for accurately predicting the risk of HFNC failure in patients with AHRF. The model had better predictive performance than the existing ROX and mROX indexes.

Trial registration: None