Revisiting Domain-Adaptive Semantic Segmentation via Knowledge Distillation.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Unsupervised Domain Adaptation (UDA) is crucial for semantic segmentation, with self-supervised methods showing promise.
• Existing self-training UDA methods risk propagating inaccuracies from pseudo-ground truths (PGTs) to the teacher model.
• Exponential Moving Average (EMA) updates in teacher models can still lead to error propagation.

Purpose of the Study:

• To propose a novel UDA approach for semantic segmentation using two distinct teacher models.
• To address the issue of inaccurate PGTs and teacher model updates in self-training UDA.
• To leverage knowledge distillation (KD) principles for improved UDA in semantic segmentation.

Main Methods:

• A novel UDA approach employing two teacher models: one EMA-updated for PGT generation and one frozen, pre-trained teacher for feature space knowledge transfer.
• Utilizing a frozen teacher model with potentially larger representation power, unconstrained by architecture.
• Revisiting self-training UDA from a KD perspective.

Main Results:

• The proposed method enhances semantic segmentation performance in target domains across various backbones and scenarios.
• Scalability demonstrated across different experimental setups (GTA5 → Cityscapes, SYNTHIA → Cityscapes).
• Achieved comparable or superior performance to state-of-the-art methods, even with lightweight backbones.

Conclusions:

• The dual-teacher model approach effectively mitigates inaccuracies in PGTs and improves teacher model updates.
• This method offers a scalable and robust solution for unsupervised domain adaptation in semantic segmentation.
• The proposed technique demonstrates the potential of KD-inspired strategies for advancing UDA research.