Automated Classification of Clinical Incident Types

Area of Science:

• Health Informatics
• Machine Learning in Healthcare
• Clinical Data Analysis

Background:

• Clinical incident reports are crucial for patient safety and quality improvement.
• Effective classification of these reports aids in identifying trends and preventing adverse events.
• Current methods for analyzing large volumes of incident reports can be time-consuming and subjective.

Purpose of the Study:

• To evaluate the effectiveness of machine learning algorithms for automatic classification of clinical incident reports.
• To compare the performance of different classification algorithms and assess the impact of data labeling expertise.
• To identify key factors influencing classification accuracy, such as class definition and data preprocessing.

Main Methods:

• Utilized a dataset of 5448 clinical incident reports from New South Wales hospitals.
• Evaluated four machine learning algorithms: decision tree, naïve Bayes, multinomial naïve Bayes, and support vector machine.
• Compared classification performance using data labeled by clinicians versus experts, and analyzed the impact of class reduction.

Main Results:

• Multinomial Naive Bayes achieved the highest accuracy (80.44%) and AUC (0.91) with initial class labeling.
• Expert-labeled data improved all classifiers' performance, with multinomial Naive Bayes reaching 81.32% accuracy and 0.97 AUC.
• Removing poorly defined classes (e.g., Primary Care) enhanced classifier performance, while some classes (e.g., Aggression Victim) were easier to classify than others (e.g., Behavior).

Conclusions:

• Machine learning, particularly multinomial Naive Bayes, is effective for classifying clinical incident reports.
• Expert class labeling and refining class definitions significantly improve classification accuracy.
• The study highlights the potential for automated systems to enhance patient safety through efficient analysis of incident data.