Machine learning and deep learning for classifying the justification of brain CT referrals

Area of Science:

• Medical Imaging Informatics
• Artificial Intelligence in Healthcare
• Radiology Workflow Optimization

Background:

• Interpreting unstructured clinical indications for radiology referrals is complex and prone to human variability.
• Current justification analysis relies on human experts, leading to potential inconsistencies and inefficiencies.
• The iGuide categorization system provides a framework for assessing referral appropriateness.

Purpose of the Study:

• To train machine learning (ML) and deep learning (DL) models for automated justification analysis of radiology referrals.
• To evaluate the models' ability to generalize across different clinical sites.
• To compare the performance of AI models against human expert judgment.

Main Methods:

• Retrospective collection of 3000 adult brain CT referrals from three Irish centers (2020-2021).
• Analysis of referral justification by radiographers and consultant radiologists using iGuide criteria.
• Application of ML and DL techniques (gradient-boosting, bi-directional LSTM) to unstructured clinical indications for multi-class classification.

Main Results:

• A gradient-boosting ML model achieved 94.4% accuracy and a 0.94 macro F1 score.
• Deep learning models, including bi-directional LSTM, showed slightly lower performance (92.3% accuracy, 0.92 macro F1).
• Human expert agreement was moderate (radiographers κ=0.268, radiologists κ=0.460), indicating significant variability.

Conclusions:

• Machine and deep learning models can effectively automate radiology referral justification, generalizing across clinical sites.
• AI-based interpretation of clinical indications can outperform human experts in retrospective vetting.
• Implementing AI systems can improve adherence to imaging referral guidelines, reduce radiation dose, and optimize resource allocation.