RCKD: Response-Based Cross-Task Knowledge Distillation for Pathological Image Analysis.

Area of Science:

• Computational pathology
• Artificial intelligence in medicine
• Medical image analysis

Background:

• Pathological image analysis requires robust models for accurate diagnosis.
• Existing transfer learning methods have limitations in cross-task and cross-architecture knowledge transfer.
• High-resolution pathological images pose computational challenges.

Purpose of the Study:

• To introduce a novel transfer learning framework, Response-based Cross-task Knowledge Distillation (RCKD), for pathological image analysis.
• To develop a lightweight neural network architecture, Convolutional neural network with Spatial Attention by Transformers (CSAT), for efficient processing of high-resolution pathological images.
• To improve the performance of models on downstream tasks like cancer sub-type classification and segmentation.

Main Methods:

• Pretraining a student model on unlabeled pathological images using teacher model predictions via RCKD.
• Fine-tuning the pretrained model on downstream tasks (classification, segmentation) with small target datasets.
• Proposing and evaluating the CSAT architecture for efficient high-resolution image processing.

Main Results:

• RCKD enables knowledge transfer across different tasks and model architectures.
• The CSAT architecture achieves high accuracy on ImageNet with minimal parameters (78.6% top-1 accuracy, 3M parameters).
• RCKD-pretrained CSAT significantly outperforms EfficientNet-B0 and ConvNextV2-Atto on pathological image datasets, achieving 94.2% classification accuracy and 0.673 mIoU segmentation accuracy.

Conclusions:

• RCKD is an effective transfer learning framework for pathological image analysis, outperforming conventional methods.
• The CSAT architecture offers an efficient solution for analyzing high-resolution pathological images.
• The combined RCKD and CSAT approach demonstrates superior performance in cancer classification and segmentation tasks.