RadBERT-CL: Factually-Aware Contrastive Learning For Radiology Report Classification.

Area of Science:

• Medical Informatics
• Natural Language Processing
• Artificial Intelligence

Background:

• Radiology reports are unstructured text containing crucial clinical information.
• Accurate extraction of diagnoses and findings is vital for patient care and research.
• Current methods struggle with factual and uncertain statements, leading to errors.

Purpose of the Study:

• To develop an advanced model for extracting information from radiology reports.
• To address limitations of existing rule-based and transformer-based approaches.
• To improve the accuracy of multi-class, multi-label report classification.

Main Methods:

• Introduced three novel augmentation techniques for contrastive learning.
• Developed RadBERT-CL, fusing information into BlueBert using self-supervised contrastive loss.
• Utilized the MIMIC-CXR dataset for experimental validation.

Main Results:

• RadBERT-CL demonstrated superior performance in fine-tuning for report classification.
• Outperformed standard transformers (BERT/BlueBert) by 6-11% with limited labeled data.
• Learned representations captured critical medical information effectively.

Conclusions:

• RadBERT-CL offers a significant advancement in analyzing unstructured radiology data.
• The model enhances diagnostic accuracy and disease monitoring capabilities.
• Effective for scenarios with scarce labeled data, improving clinical insights.