A Multi-Teacher Knowledge Distillation Framework for Enhancing the Robustness of Automated Sperm Morphology...

Area of Science:

• * Computational biology and medical image analysis.
• * Application of deep learning in andrology.

Background:

• * Manual sperm morphology analysis is subjective and time-consuming, impacting male infertility diagnosis.
• * Deep learning models struggle with overfitting on small, imbalanced clinical datasets, limiting generalization.
• * Knowledge distillation is proposed to enhance the robustness of automated sperm morphology analysis.

Purpose of the Study:

• * To investigate a knowledge distillation approach for improving automated sperm morphology analysis.
• * To evaluate single-teacher versus multi-teacher distillation strategies.
• * To address challenges of data scarcity and class imbalance in clinical datasets.

Main Methods:

• * Soft knowledge distillation from high-capacity teacher models (SwinV2-large, EfficientNetV2-m, ConvNeXtV2-large) to a smaller student model (SwinV2-base).
• * Training on the imbalanced Hi-LabSpermMorpho dataset with 18 morphology categories across three staining methods.
• * Cross-dataset training and a combined loss function (cross-entropy and KL divergence) to regularize the student model.

Main Results:

• * The multi-teacher distillation setup achieved higher accuracies: 70.94% (BesLab), 73.61% (Histoplus), 71.63% (GBL).
• * The student model demonstrated improved generalization compared to baseline models.
• * Soft distillation effectively prevented student model over-confidence.

Conclusions:

• * The proposed knowledge distillation approach effectively mitigates overfitting and improves robustness.
• * This method offers a highly generalizable solution for automated sperm morphology analysis in clinical diagnostics.
• * Consistent performance improvements were observed across different staining methods and datasets.