Automatic Incident Triage in Radiation Oncology Incident Learning System

Area of Science:

• Medical Informatics
• Computational Linguistics
• Radiation Oncology

Background:

• The Radiotherapy Incident Reporting and Analysis System (RIRAS) collects incident data from Radiation Oncology facilities.
• Manual review of incident reports by subject matter experts (SMEs) is time-consuming.
• Automating the triage and severity assessment of these reports is crucial for efficient safety management.

Purpose of the Study:

• To develop and evaluate a computational pipeline using machine learning (ML) and natural language processing (NLP) to predict the severity of radiotherapy incidents.
• To automate the triage process for incident reports within the RIRAS platform.
• To compare the performance of traditional ML algorithms against transfer learning approaches for severity prediction.

Main Methods:

• Utilized NLP and ML techniques to analyze textual descriptions in RIRAS incident reports.
• Developed models to classify incident severity into High (A & B) vs. Low (C & D) categories.
• Compared a Support Vector Machine (SVM) with linear kernel against the Universal Language Model Fine-Tuning (ULMFiT) transfer learning algorithm.

Main Results:

• The SVM-linear model achieved an F1-Score of 0.78 on the VHA dataset but 0.5 on the VCU dataset.
• The ULMFiT transfer learning approach demonstrated better performance, achieving 0.81 F1-Score on the VHA dataset and 0.68 on the VCU dataset.
• Transfer learning showed more consistent performance across different datasets.

Conclusions:

• The proposed computational pipeline shows significant promise for automating the triage and severity determination of radiotherapy incidents.
• NLP and ML methods, particularly transfer learning, can enhance the efficiency and accuracy of incident report analysis.
• Further development of these automated methods can improve patient safety in radiation oncology.