Knowledge Distillation in Histology Landscape by Multi-Layer Features Supervision.

Area of Science:

• Computational Pathology
• Digital Pathology
• Machine Learning in Histology

Background:

• Automatic tissue classification is crucial for understanding tumor micro-environments.
• Deep learning models offer high performance but demand substantial computational resources.
• Shallow networks, while efficient, struggle with tissue heterogeneity when trained with direct supervision.

Purpose of the Study:

• To develop a novel knowledge distillation algorithm to enhance the performance of shallow networks for tissue phenotyping in histology images.
• To address the limitations of shallow networks in capturing tissue heterogeneity.
• To improve computational efficiency in computational pathology.

Main Methods:

• Proposed a multi-layer feature distillation approach within the Knowledge Distillation for Tissue Phenotyping (KDTP) algorithm.
• Student network layers receive supervision from multiple teacher network layers.
• Feature map sizes are matched using a learnable multi-layer perceptron, minimizing feature map distance during training.
• An attention-based parameter weights the summation of multi-layer losses.

Main Results:

• The proposed KDTP algorithm demonstrated significant performance improvements in student networks across five histology image classification datasets.
• Achieved superior results compared to direct supervision-based training methods.
• Validated the effectiveness of multi-layer feature distillation and attention-based weighting.

Conclusions:

• The KDTP algorithm effectively enhances shallow networks for tissue phenotyping in computational pathology.
• This approach offers a computationally efficient method for improving histology image classification accuracy.
• KDTP provides a promising direction for advancing digital pathology tools.