Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Semantic nighttime image segmentation is challenging due to poor visibility and lack of labeled data.
• Existing daytime models perform poorly on nighttime images without adaptation.
• Substantial uncertainty exists in nighttime image semantics, complicating evaluation.

Purpose of the Study:

• To adapt daytime semantic segmentation models for nighttime conditions without using nighttime annotations.
• To develop a novel evaluation framework and metric that accounts for uncertainty in nighttime image semantics.
• To introduce the Dark Zurich dataset as a benchmark for nighttime semantic segmentation.

Main Methods:

• A curriculum framework progressively adapts models from day to night using cross-time-of-day correspondences.
• Exploiting reference maps and progressively darker times of day to guide label inference.
• Developing an uncertainty-aware annotation and evaluation framework, including regions beyond human recognition.

Main Results:

• Map-guided curriculum adaptation significantly outperforms state-of-the-art methods on nighttime datasets.
• The novel uncertainty-aware metric provides a more principled evaluation for nighttime images.
• Selective invalidation of predictions improves results on ambiguous data and benefits safety applications.

Conclusions:

• Daytime semantic segmentation models can be effectively adapted for nighttime use via a curriculum learning approach.
• The proposed uncertainty-aware evaluation framework is crucial for reliable nighttime image analysis.
• The Dark Zurich dataset and new evaluation metric establish a benchmark for future research in this domain.