Evaluating Active Learning Strategies for Automated Classification of Patient Safety Event Reports in Hospitals

Area of Science:

• Health Informatics
• Machine Learning Applications in Healthcare
• Patient Safety Research

Background:

• Patient safety event (PSE) reports are vital for healthcare quality improvement, trend analysis, and organizational learning.
• Manual classification of PSE reports is labor-intensive due to high volume and complex taxonomies.
• Machine learning (ML) for PSE report classification requires large, manually labeled datasets, posing a significant bottleneck.

Purpose of the Study:

• To investigate the efficacy of Active Learning (AL) strategies in streamlining the labeling process for PSE reports.
• To reduce the manual workload associated with creating labeled datasets for ML-based PSE report classification.
• To enhance the accuracy and efficiency of PSE report classification through optimized data annotation.

Main Methods:

• Employed pool-based Active Learning (AL) sampling to intelligently select PSE reports for human annotation.
• Developed a robust dataset for training ML classifiers by prioritizing informative samples identified by AL.
• Compared the performance of AL-based labeling against random sampling across various text representations.

Main Results:

• Active Learning (AL) significantly outperformed random sampling in classification accuracy.
• AL strategies reduced the requirement for labeled samples by 24% to 69%, demonstrating substantial efficiency gains.
• The developed approach maintained high classification accuracy while minimizing manual annotation effort.

Conclusions:

• Integrating Active Learning (AL) into the PSE report labeling workflow can substantially decrease manual workload.
• AL offers an effective method for building high-quality labeled datasets for ML models in patient safety.
• This strategy supports more efficient and accurate analysis of patient safety events, contributing to improved healthcare quality.