CReg-KD: Model refinement via confidence regularized knowledge distillation for brain imaging.

Area of Science:

• Deep Learning
• Medical Image Analysis
• Neuroimaging

Background:

• Deep learning models for 3D brain imaging face challenges due to data insufficiency, leading to overfitting and poor generalization.
• Regularization techniques, like knowledge distillation, are crucial for enhancing model performance by reinforcing training with additional knowledge.

Purpose of the Study:

• To propose and evaluate a novel Confidence Regularized Knowledge Distillation (CReg-KD) framework for medical image analysis.
• To adaptively transfer knowledge during distillation based on knowledge confidence, improving regularization.

Main Methods:

• Developed CReg-KD framework to penalize attentive output distributions and intermediate representations.
• Implemented a gated distillation mechanism using teacher loss as a global confidence score.
• Employed attentive local refinement of intermediate representations to mimic semantic features.

Main Results:

• CReg-KD demonstrated consistent improvements over baseline models on Alzheimer's Disease classification and brain age estimation.
• The framework outperformed existing state-of-the-art knowledge distillation methods.
• Evaluated on large datasets (ADNI and 4 public cohorts) including over 4500 subjects.

Conclusions:

• CReg-KD effectively mitigates data insufficiency challenges in 3D brain imaging deep learning.
• The proposed framework enhances both prediction performance and generalizability for medical image analysis.
• CReg-KD offers a powerful tool for analyzing medical imaging data, particularly in neurological applications.