Related Experiment Video
Updated: Aug 13, 2025
04:48
Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
2.9K
FCKDNet: A Feature Condensation Knowledge Distillation Network for Semantic Segmentation
Wenhao Yuan1, Xiaoyan Lu1, Rongfen Zhang1
1College of Big Data and Information Engineering, Guizhou University, Guiyang 550025, China.
Entropy (Basel, Switzerland)
|January 21, 2023
Summary
This study introduces a feature condensation method to improve knowledge distillation (KD) for semantic segmentation (SS). The proposed FCKDNet enhances teacher network features, reducing noise and boosting student model performance.
Area of Science:
- Computer Vision
- Machine Learning
Background:
- Knowledge distillation (KD) is crucial for semantic segmentation (SS) in computer vision.
- Current KD methods are limited by the quality of feature knowledge from teacher networks.
Purpose of the Study:
- To propose a novel feature condensation-based knowledge distillation network (FCKDNet).
- To reduce pseudo-knowledge transfer and improve feature representation in teacher-student networks for semantic segmentation.
Main Methods:
- Developed a feature condensation method using pixel information entropy to separate foreground features from background noise.
- Applied a feature condensation matrix to teacher and student network outputs to enhance feature representation.
- Introduced a soft feature enhancement method across spatial and channel dimensions.
- Implemented separate distillation loss calculations for spatial and channel condensation features.
Main Results:
- FCKDNet improved baseline performance by 3.16% (mAcc) on Pascal VOC and 2.98% (mAcc) on Cityscapes.
- Achieved 2.03% (mIoU) and 2.30% (mIoU) improvements on Pascal VOC and Cityscapes, respectively.
- Demonstrated superior segmentation performance and robustness compared to mainstream methods.
Conclusions:
- The proposed feature condensation technique effectively mitigates pseudo-knowledge transfer in KD for SS.
- FCKDNet enhances feature representation and accelerates student network convergence.
- The method offers improved segmentation accuracy and robustness in computer vision applications.
