Learning Domain-Invariant Representations for Event-Based Motion Segmentation: An Unsupervised Domain Adaptation...

Area of Science:

• Computer Vision
• Robotics
• Sensor Fusion

Background:

• Event cameras offer high temporal resolution and dynamic range, ideal for high-speed applications.
• Sparsity of event data and lack of annotations hinder supervised learning for motion segmentation.
• Domain adaptation is difficult due to significant shifts between intensity images and event data.

Purpose of the Study:

• To develop a cross-modality adaptation framework for motion segmentation from event streams.
• To transfer knowledge from labeled RGB-flow data to unlabeled event data.
• To address challenges of data sparsity, annotation scarcity, and domain shift in event-based vision.

Main Methods:

• A dual-branch encoder extracts features from RGB and optical flow in the source domain.
• Reconstruction networks convert event data into pseudo-image and pseudo-flow modalities for the target domain.
• Multi-level consistency losses enforce domain alignment on features, predictions, and outputs.

Main Results:

• The proposed framework achieves 83.1% accuracy on EVIMO2 and 79.4% on MOD++.
• Outperforms existing methods like EV-Transfer and SHOT by up to 3.6%.
• Demonstrates a lightweight architecture enabling real-time inference with enhanced mIoU and F1 Score.

Conclusions:

• The cross-modality adaptation framework effectively bridges the domain and modality gap for event-based motion segmentation.
• The method enables acquisition of domain-invariant, semantically rich features with reduced training costs.
• The approach is suitable for real-world high-speed applications like autonomous driving.