Doctors Identify Hemorrhage Better during Chart Review when Assisted by Artificial Intelligence

Area of Science:

• Clinical Informatics
• Medical Artificial Intelligence
• Health Informatics

Background:

• Hemorrhage detection in electronic health records (EHRs) is crucial for patient care.
• Manual chart review is time-consuming and prone to overlooking critical information.
• Existing methods for hemorrhage identification lack sufficient accuracy and efficiency.

Purpose of the Study:

• To evaluate the impact of an artificial intelligence (AI) model on medical doctors' ability to identify hemorrhage events during chart review.
• To assess medical doctors' perceptions of using an AI model as a clinical support tool.
• To analyze the reading workflow of medical doctors during manual and AI-assisted chart review.

Main Methods:

• Developed an AI model by labeling hemorrhage-related sentences from 900 EHRs across 12 anatomical locations.
• Evaluated the AI model's performance on a test cohort of 566 admissions (sensitivity: 93.7%, specificity: 98.1%).
• Conducted a clinical use study using eye-tracking to analyze reading workflows and comparing chart review performance with and without AI assistance.

Main Results:

• Medical doctors missed over 33% of relevant hemorrhage sentences during unassisted chart review.
• AI-assisted chart review led to a 48-49 percentage point increase in hemorrhage event identification.
• Hemorrhage events in paragraphs were more frequently overlooked than bullet-pointed mentions.

Conclusions:

• AI-assisted chart review significantly enhances medical doctors' accuracy in identifying hemorrhage events.
• Medical doctors generally reported positive perceptions of using AI as a supportive tool in clinical practice.
• The AI model demonstrates high performance, improving diagnostic efficiency and potentially patient outcomes.