SELF-SUPERVISED LEARNING WITH RADIOLOGY REPORTS, A COMPARATIVE ANALYSIS OF STRATEGIES FOR LARGE VESSEL OCCLUSION AND...

Area of Science:

• Medical imaging
• Deep learning
• Natural language processing

Background:

• Label scarcity hinders deep learning model training for medical images.
• Radiology reports accompanying medical images offer valuable information for pre-training.
• Self-supervised learning can leverage unlabeled data for model pre-training.

Purpose of the Study:

• To compare three self-supervised strategies for pre-training imaging models on 3D brain computed tomography angiogram (CTA) images.
• To evaluate the effectiveness of natural language processing (NLP) approaches for extracting information from radiology reports.
• To assess the performance improvement for large vessel occlusion (LVO) detection using pre-trained models.

Main Methods:

• Pre-training on 1,542 unlabeled 3D CTA-report pairs using three self-supervised strategies.
• Evaluating NLP approaches: Rad-SpatialNet for explicit concepts and DistilBERT for report embeddings.
• Utilizing a contrastive language-image pre-training (CLIP) approach for learning joint representations.
• Fine-tuning and testing on a labeled dataset of 402 subjects for LVO detection.

Main Results:

• CLIP-based pre-training strategies improved imaging model performance for LVO detection compared to training solely on labeled data.
• Pre-training using explicit radiology concepts combined with CLIP yielded the best performance.
• The study demonstrated the potential of leveraging radiology reports for enhancing medical image analysis.

Conclusions:

• Self-supervised pre-training using radiology reports is a viable strategy to overcome label scarcity in medical imaging.
• Integrating explicit radiology concepts with CLIP-based learning offers a powerful approach for downstream medical image analysis tasks like LVO detection.
• Future work can explore further optimization of NLP and self-supervised techniques for medical applications.