Development and external validation of a machine learning model to predict high flow nasal cannula failure

High-flow nasal cannula (HFNC) is an important treatment option for acute hypoxic respiratory failure and can improve outcomes. However, patients on a prolonged duration of HFNC have worse clinical outcomes and increased mortality. It is difficult to determine which patients will fail HFNC support at the time of initiation.

Does an externally validated machine learning model predict HFNC failure with greater discrimination compared with the ROX Index?

Adult inpatients hospitalised at seven hospitals in four health systems who received HFNC were eligible for inclusion. Patients were excluded if they were intubated before first HFNC initiation, experienced the primary outcome <1 hour after HFNC initiation, unable to calculate the ROX Index or began HFNC or experienced intubation or death in a location other than the studied locations. A gradient boosting model was used to predict the primary composite outcome of intubation or death within the next 24 hours at the time of HFNC initiation. The model was compared with the previously published ROX index.

Of the 11 618 patients included in the study, 6787 were in the training cohort and 4831 were in the testing cohort. The primary outcome occurred in 1410 of 11 618 (12.1%) patients at 24 hours. In external validation, the area under the operating curve of the model for predicting HFNC failure within 24 hours was 0.760, which was significantly higher than the ROX index (0.696; p value <0.001). This improvement in performance was consistent at all studied time periods, including 2, 6 and 12 hours after HFNC initiation.

In this study, we developed and externally validated a novel machine learning algorithm that outperforms the ROX Index in predicting failure of HFNC in patients with acute hypoxic respiratory failure. This model could augment clinical decision-making when treating patients with this morbid condition.