Evaluating an AI Decision Support System for the Emergency Department: Retrospective Study

Area of Science:

• Medical Informatics
• Clinical Decision Support Systems
• Artificial Intelligence in Healthcare

Background:

• Emergency department (ED) overcrowding is a critical issue linked to increased medical errors, prolonged patient stays, and higher mortality rates.
• Artificial intelligence (AI) decision support tools offer potential solutions for optimizing patient flow and decision-making in EDs.
• Existing research often overlooks the clinical relevance and practical implementation of AI in healthcare settings.

Purpose of the Study:

• To evaluate the clinical utility of an AI model in predicting patient admissions from the ED.
• To assess the potential of AI to reduce the time required for making admission decisions.
• To investigate the impact of AI on alleviating ED overcrowding and improving patient care.

Main Methods:

• A retrospective analysis of 154,347 anonymized patient visits from St. Antonius Hospital (January 2018 - September 2023).
• Development and testing of an Extreme Gradient Boosting AI model to predict hospital admission decisions.
• Evaluation of the AI model using data segmented into 10-minute intervals to simulate real-world applicability and measure decision-making time reduction.

Main Results:

• The AI model achieved a precision of 0.78 and a recall of 0.73.
• A median time saving of 111 minutes (IQR 59-169) was observed for patients where the AI correctly predicted admission.
• Subgroup analyses indicated greater time savings for older patients and in specialties like pulmonology, with some cases saving up to 90 minutes per patient.

Conclusions:

• The developed AI model demonstrates significant potential in reducing ED admission decision times, thereby mitigating overcrowding.
• The AI tool provides consistent, weighted admission advice, even during periods of high ED pressure.
• Further prospective studies are recommended to validate the real-world impact and optimize the AI model across diverse clinical environments.