Physician Gestalt Compared With Artificial Intelligence Model to Predict Intubation in Critically Ill Patients

Area of Science:

• Critical Care Medicine
• Artificial Intelligence in Healthcare
• Clinical Decision Support

Background:

• Accurate prediction of endotracheal intubation is crucial for improving outcomes in critically ill patients.
• The comparative performance of intensive care physicians and machine learning (ML) models in predicting intubation needs is not well-established.

Purpose of the Study:

• To evaluate the accuracy of intensive care physicians in predicting the need for intubation within 24 hours.
• To compare physician prediction performance against the established ML model, Vent.io.

Main Methods:

• A prospective observational study involving intensivists predicting intubation needs in adult ICU patients.
• Physician predictions were compared with Vent.io predictions using sensitivity, specificity, and generalized linear mixed models.
• Physician confidence in predictions was also analyzed for its association with accuracy.

Main Results:

• Physicians demonstrated high confidence (median score 8/10) but lower sensitivity (0.190) compared to Vent.io (0.714) for predicting intubation.
• Physicians showed higher specificity (0.960) than Vent.io (0.673) in predicting no need for intubation.
• Vent.io was significantly better at predicting the need for intubation, while physicians excelled at predicting when intubation was not required.

Conclusions:

• The Vent.io ML model shows promising predictive performance compared to human experts in critical care settings.
• Further real-time testing via randomized clinical trials is necessary to confirm if Vent.io deployment can enhance patient clinical outcomes.
• Physician confidence correlates with prediction accuracy, but ML models may offer superior sensitivity for intubation prediction.