Interactive Learning of Intrinsic and Extrinsic Properties for All-Day Semantic Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Current semantic segmentation models struggle with significant appearance changes throughout the day.
• Existing domain adaptation methods lack generalization for all-day scenarios due to fixed mapping limitations.

Purpose of the Study:

• To develop a robust semantic segmentation method capable of handling drastic appearance variations from dawn to night.
• To improve the generalization capability of segmentation models for all-day environmental conditions.

Main Methods:

• Proposed a novel intrinsic-extrinsic interactive learning strategy to disentangle image appearance into stable intrinsic and dynamic extrinsic representations.
• Introduced an All-in-One Segmentation Network (AO-SegNet) for end-to-end training.
• Utilized spatial-wise guidance for interaction between intrinsic and extrinsic representations.

Main Results:

• AO-SegNet demonstrated significant performance gains over state-of-the-art methods on multiple real-world datasets (Mapillary, BDD100K, ACDC) and a synthetic dataset.
• The intrinsic-extrinsic interactive learning strategy led to more stable intrinsic representations and better extrinsic change depiction.
• The method showed robustness across various Convolutional Neural Network (CNN) and Vision Transformer (ViT) backbones.

Conclusions:

• The proposed intrinsic-extrinsic interactive learning strategy effectively addresses the challenge of semantic segmentation in all-day scenarios.
• AO-SegNet provides a robust and generalizable solution for semantic segmentation under varying illumination conditions.
• This approach enhances the reliability of pixel-wise predictions in dynamic environmental settings.