A Machine Learning Approach with Human-AI Collaboration for Automated Classification of Patient Safety Event Reports:...

Area of Science:

• Natural Language Processing
• Machine Learning
• Patient Safety

Background:

• Patient safety event (PSE) reports are crucial for monitoring hospital incidents but face classification challenges due to volume and inconsistency.
• Transformer-based language models offer advanced contextual text representation for more precise PSE report classification.
• Integrating machine learning (ML) with human expertise requires explainability for effective human-AI collaboration.

Purpose of the Study:

• Investigate the efficacy of ML classifiers trained with contextual text representation for automatic PSE report classification.
• Present an interface integrating ML classifiers with explainability techniques to foster human-AI collaboration in PSE report classification.

Main Methods:

• Utilized 861 PSE reports from a Southeastern US academic hospital's maternity units.
• Trained and evaluated ML classifiers using both static and contextual text representations of PSE reports.
• Employed the Local Interpretable Model-Agnostic Explanations (LIME) technique for prediction rationale and designed an integrated reporting interface.

Main Results:

• The top-performing contextual representation classifier achieved 75.4% accuracy, outperforming static representation classifiers (66.7%).
• A PSE reporting interface was developed, recommending top event classifications with explanations to aid user selection.
• LIME analysis revealed the classifier's occasional reliance on arbitrary words, underscoring the need for human oversight.

Conclusions:

• Contextual text representations significantly enhance ML-based PSE report classification accuracy.
• The developed interface supports human-AI collaboration, improving decision-making for patient safety.
• This research enables more efficient risk identification and timely interventions to prevent patient harm.