Comparison of Supervised and Self-Supervised Deep Representations Trained on Histological Images

Area of Science:

• Medical imaging analysis
• Machine learning interpretability
• Computer vision

Background:

• Self-supervised learning (SSL) is increasingly vital in medical AI due to limited labeled data.
• SSL models achieve performance comparable or superior to supervised methods.
• The specific information encoded by SSL versus supervised models remains poorly understood.

Purpose of the Study:

• To introduce a novel framework for comparing information coding in supervised and self-supervised models.
• To explain the differences in learned visual features relevant to human perception.
• To analyze these differences in the context of medical image analysis for Gleason scoring.

Main Methods:

• Development of a new comparative framework focusing on human perceptual visual characteristics.
• Application of the framework to analyze models trained for Gleason score prediction.
• Evaluation of information coding related to texture, contrast, and shape.

Main Results:

• Self-supervised models demonstrated a stronger bias towards encoding contrast and texture information.
• Supervised models were found to encode more information related to object shape.
• The framework successfully highlighted distinct information processing strategies between the two learning paradigms.

Conclusions:

• Significant differences exist in the visual information encoded by self-supervised and supervised models in medical imaging.
• Self-supervised methods excel at capturing textural and contrastual features, crucial for certain diagnostic tasks.
• Supervised methods retain a stronger capacity for shape-based feature extraction, potentially important for other medical applications.