Measuring Domain Shift for Deep Learning in Histopathology

Area of Science:

• Digital pathology
• Deep learning
• Computer vision

Background:

• Deep learning models excel at fitting data but struggle with generalization due to domain shift.
• Domain shift, differences in image statistics between training and test data, is prevalent in digital pathology (e.g., whole-slide images from different sources).
• Understanding and quantifying domain shift is crucial for reliable deployment of deep learning in histopathology.

Purpose of the Study:

• To develop a novel measure, "representation shift," for quantifying model-specific domain shift in deep learning.
• To assess the correlation of representation shift with performance drops across various domain shifts.
• To improve upon existing methods for measuring data shift and uncertainty in digital pathology.

Main Methods:

• Focus on the internal representations learned by convolutional neural networks.
• Formulated a novel measure: representation shift.
• Studied domain shift in tumor classification using hematoxylin and eosin stained images, varying datasets, models, and data preparation techniques.

Main Results:

• The proposed representation shift measure strongly correlates with performance degradation across diverse domain shifts.
• Representation shift outperforms existing techniques for measuring data shift and uncertainty.
• The measure effectively reveals model sensitivity to domain variations and identifies data likely to cause generalization problems.

Conclusions:

• Representation shift is a valuable tool for understanding and quantifying model-specific domain shift in digital pathology.
• This measure aids in detecting data on which models may fail to generalize, enhancing reliability.
• Addressing domain shift through measurement and mitigation is key for the clinical adoption of deep learning in pathology.