Related Experiment Video
Updated: May 10, 2025
04:48
Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
2.6K
Feature distance-weighted adaptive decoupled knowledge distillation for medical image segmentation
Xiangchun Yu1, Ziyun Xiong1, Miaomiao Liang2
1Jiangxi Provincial Key Laboratory of Multidimensional Intelligent Perception and Control, School of Information Engineering, Jiangxi University of Science and Technology, Ganzhou, 341000, China.
Summary
This study introduces feature distance-weighted adaptive decoupled knowledge distillation (FDWA-DKD) for medical image segmentation. Our method enhances knowledge transfer from teacher to student networks, improving performance on embedded devices.
Area of Science:
- Artificial Intelligence
- Computer Vision
- Medical Imaging
Background:
- Deep learning models for medical image segmentation often require significant computational resources, hindering deployment on resource-constrained embedded devices.
- Knowledge distillation offers a way to transfer knowledge from large, high-performance models to smaller, more efficient ones.
Purpose of the Study:
- To apply decoupled knowledge distillation (DKD) to medical image segmentation, enabling the deployment of lightweight student networks on embedded devices.
- To improve the efficiency and effectiveness of knowledge transfer in medical image segmentation models.
Main Methods:
- Decoupled the distillation loss into pixel-wise target class knowledge distillation (PTCKD) and pixel-wise non-target class knowledge distillation (PNCKD).
- Proposed a novel feature distance-weighted adaptive decoupled knowledge distillation (FDWA-DKD) method to address limitations of fixed weights in PTCKD.
- Introduced a feature distance weighting (FDW) module for instance-level adaptive weights and a class-wise feature probability distribution loss.
Main Results:
- FDWA-DKD achieved optimal Dice scores on Synapse and FLARE22 datasets.
- The proposed method demonstrated performance comparable to, and in some cases exceeding, the teacher network.
- Ablation studies confirmed the effectiveness of individual modules within the FDWA-DKD framework.
Conclusions:
- FDWA-DKD overcomes traditional distillation constraints by providing instance-level adaptive weights for PTCKD.
- Quantifying student-teacher feature disparity and minimizing class-wise feature probability distribution loss leads to superior performance.
- The method offers a viable solution for deploying high-performance medical image segmentation models on embedded systems.
