Early Warning Scores With and Without Artificial Intelligence.

Area of Science:

• Clinical Informatics
• Artificial Intelligence in Healthcare
• Patient Safety

Background:

• Early warning scores (EWS) are crucial for detecting clinical deterioration in hospitalized patients.
• Existing EWS have varying comparative performance, necessitating further evaluation.

Purpose of the Study:

• To compare the performance of three proprietary artificial intelligence (AI) EWS against three publicly available simple aggregated weighted scores.
• To assess the accuracy and lead time provided by different EWS in identifying patient deterioration.

Main Methods:

• A retrospective cohort study included over 360,000 adult medical-surgical ward encounters across seven hospitals.
• Six EWS were evaluated: Epic Deterioration Index (EDI), Rothman Index (RI), eCARTv5 (eCART), Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), and NEWS2.
• Clinical deterioration was defined as transfer to ICU or death within 24 hours.

Main Results:

• eCART achieved the highest area under the receiver operating characteristic curve (0.895), indicating superior discrimination.
• NEWS2 and NEWS also showed strong performance, outperforming EDI, RI, and MEWS.
• At moderate-risk thresholds, all scores provided a median of at least 20 hours of lead time; eCART offered the longest median lead time (11 hours) at high-risk thresholds.

Conclusions:

• The eCART AI score demonstrated superior accuracy and provided a longer lead time for detecting clinical deterioration compared to other evaluated scores.
• Publicly available scores like NEWS also showed significant performance, outperforming some proprietary AI tools.
• The findings suggest a need for greater transparency and oversight in the development and implementation of EWS.