BERT-based Transfer Learning in Sentence-level Anatomic Classification of Free-Text Radiology Reports
Daiki Nishigaki1, Yuki Suzuki1, Tomohiro Wataya1
1Departments of Artificial Intelligence Diagnostic Radiology (D.N., Y.S., T.W., K.K., K.Y., J.S., S.K.) and Radiology (N.T.), Osaka University Graduate School of Medicine, 2-2 Yamadaoka, Suita, Osaka 565-0871, Japan.
Purpose:
To assess whether transfer learning with a bidirectional encoder representations from transformers (BERT) model, pretrained on a clinical corpus, can perform sentence-level anatomic classification of free-text radiology reports, even for anatomic classes with few positive examples.
Materials And Methods:
This retrospective study included radiology reports of patients who underwent whole-body PET/CT imaging from December 2005 to December 2020. Each sentence in these reports (6272 sentences) was labeled by two annotators according to body part ("brain," "head & neck," "chest," "abdomen," "limbs," "spine," or "others"). The BERT-based transfer learning approach was compared with two baseline machine learning approaches: bidirectional long short-term memory (BiLSTM) and the count-based method. Area under the precision-recall curve (AUPRC) and area under the receiver operating characteristic curve (AUC) were computed for each approach, and AUCs were compared using the DeLong test.
Results:
The BERT-based approach achieved a macro-averaged AUPRC of 0.88 for classification, outperforming the baselines. AUC results for BERT were significantly higher than those of BiLSTM for all classes and those of the count-based method for the "brain," "chest," "abdomen," and "others" classes (P values < .025). AUPRC results for BERT were superior to those of baselines even for classes with few labeled training data (brain: BERT, 0.95, BiLSTM, 0.11, count based, 0.41; limbs: BERT, 0.74, BiLSTM, 0.28, count based, 0.46; spine: BERT, 0.82, BiLSTM, 0.53, count based, 0.69).
Conclusion:
The BERT-based transfer learning approach outperformed the BiLSTM and count-based approaches in sentence-level anatomic classification of free-text radiology reports, even for anatomic classes with few labeled training data.Keywords: Anatomy, Comparative Studies, Technology Assessment, Transfer Learning Supplemental material is available for this article. © RSNA, 2023.
