Domain-adapted Large Language Models for Classifying Nuclear Medicine Reports.

Area of Science:

• Artificial Intelligence in Medical Imaging
• Natural Language Processing (NLP) for Clinical Reports
• Machine Learning in Nuclear Medicine

Background:

• Accurate prediction of Deauville scores from PET/CT reports is crucial for lymphoma treatment assessment.
• Existing language models may struggle with the specialized terminology and context of nuclear medicine reports.
• Domain adaptation offers a potential method to improve NLP model performance on specific medical tasks.

Purpose of the Study:

• To evaluate the impact of domain adaptation on the performance of language models in predicting five-point Deauville scores.
• To compare domain-adapted language models against non-domain-adapted versions, vision models, multimodal models, and human expert performance.

Main Methods:

• Retrospective analysis of 4542 PET/CT lymphoma examination reports, with 1664 Deauville scores extracted for training.
• Domain adaptation of BERT, BioClinicalBERT, RadBERT, and RoBERTa models using masked language modeling on nuclear medicine data.
• Comparison of models using sevenfold Monte Carlo cross-validation for five-point Deauville score prediction.

Main Results:

• Domain adaptation significantly improved the accuracy of all tested language models (P = .01).
• Domain-adapted RoBERTa achieved the highest accuracy (77.4% ± 3.4), outperforming vision-only models and a nuclear medicine physician (66% accuracy).
• The best-performing domain-adapted RoBERTa model performed comparably to its multimodal counterpart.

Conclusions:

• Domain adaptation is an effective strategy for enhancing the performance of large language models in predicting Deauville scores from PET/CT reports.
• AI models, particularly domain-adapted RoBERTa, demonstrate potential to match or exceed human expert performance in this task.
• This research highlights the value of transfer learning and NLP in improving the analysis of clinical imaging reports.