Machine learning based natural language processing of radiology reports in orthopaedic trauma

Area of Science:

• Medical Informatics
• Artificial Intelligence in Radiology
• Natural Language Processing

Background:

• Accurate classification of radiology reports is crucial for identifying injuries in orthopaedic trauma.
• Natural Language Processing (NLP) methods are increasingly used to automate report analysis.
• Evaluating different NLP techniques is essential for clinical and research applications.

Purpose of the Study:

• To compare the performance of various Machine Learning (ML) and NLP methods for classifying injuries in orthopaedic trauma radiology reports.
• To assess the effectiveness of different preprocessing steps and classifiers, including Bidirectional Encoder Representations from Transformers (BERT).

Main Methods:

• Utilized datasets of Dutch radiology reports for injured extremities and chest radiographs from two hospitals.
• Reports were labeled by radiologists and trauma surgeons to identify the presence or absence of injuries.
• Compared Rule-based, ML, and BERT classifiers, evaluating performance using F1-score, AUC, sensitivity, specificity, and accuracy.

Main Results:

• The BERT model significantly outperformed other classification methods.
• On simple reports (n=2469), BERT achieved an F1-score of (95 ± 2)% and accuracy of (96 ± 1)%.
• On complex reports (n=799), BERT achieved an F1-score of (83 ± 7)% and accuracy of (93 ± 2)%.

Conclusions:

• BERT-based NLP demonstrates superior performance in classifying injuries within Dutch orthopaedic trauma radiology reports.
• This deep learning approach surpasses traditional ML and rule-based classifiers for this specific task.
• Effective NLP classification is vital for improving clinical workflows and research in trauma radiology.