Generalization of vision pre-trained models for histopathology

Area of Science:

• Computer Science
• Medical Imaging
• Machine Learning

Background:

• Out-of-distribution (OOD) generalization is a critical challenge in machine learning, particularly for medical applications.
• Evaluating pre-trained models on unseen histopathology data from different trial sites is essential for robust AI development.

Purpose of the Study:

• To investigate the OOD generalization performance of various pre-trained convolutional models on histopathology datasets.
• To compare models pre-trained on natural images (ImageNet, SSL, SWSL) with models pre-trained on histopathology data (KimiaNet).

Main Methods:

• Examined OOD performance across different trial site repositories, pre-trained models, and image transformations.
• Compared models trained from scratch versus pre-trained models.
• Evaluated vanilla ImageNet, semi-supervised learning (SSL), and semi-weakly-supervised learning (SWSL) models, alongside a histopathology-specific model (KimiaNet).

Main Results:

• Histopathology pre-trained models demonstrated superior OOD performance compared to general natural image pre-trained models.
• SSL and SWSL models showed better OOD generalization than vanilla ImageNet models.
• Image diversification through transformations improved top-1 accuracy by mitigating shortcut learning during significant distribution shifts.

Conclusions:

• Pre-training on domain-specific data (histopathology) is crucial for effective OOD generalization in medical AI.
• Image augmentation techniques are vital for enhancing model robustness against distribution shifts.
• Explainable AI (XAI) techniques can further aid in understanding model decisions for medical applications.