Online Self-Training Driven Attention-Guided Self-Mimicking Network for Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Knowledge distillation (KD) is key for semantic segmentation, transferring knowledge from large to small networks.
• Current KD methods often require complex, large teacher networks, complicating training.

Purpose of the Study:

• To develop a novel, efficient knowledge distillation method for semantic segmentation.
• To reduce reliance on cumbersome teacher networks in the knowledge distillation process.

Main Methods:

• Introduced a self-training driven attention-guided self-mimicking online ensemble network (ST-ASMNet).
• Utilized intermediate channel-joint attention maps for guided image augmentation.
• Employed knowledge distillation with self-training and an exponential moving average (EMA)-teacher network.

Main Results:

• Validated the effectiveness of ST-ASMNet on benchmark datasets (Cityscapes, Pascal VOC, CamVid, ADE20k).
• Demonstrated improved performance in semantic segmentation tasks.
• Showcased the interpretability of the proposed method through visualization analyses.

Conclusions:

• ST-ASMNet offers an effective and interpretable approach to knowledge distillation for semantic segmentation.
• The method enhances student network performance by learning from credible predictions and invariant features.
• The proposed technique simplifies the training process compared to existing knowledge distillation methods.