SeCoV2: Semantic Connectivity-Driven Pseudo-Labeling for Robust Cross-Domain Semantic Segmentation

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Pseudo-labeling is key for cross-domain semantic segmentation (CDSS) but struggles with noisy predictions under domain shifts.
• Existing methods often produce fragmented and inaccurate pixel-level labels, limiting performance.

Purpose of the Study:

• To introduce a novel pseudo-labeling framework, SeCo, that leverages semantic connectivity for improved CDSS.
• To present SeCoV2, an enhanced version addressing ambiguity and expanding applicability to challenging scenarios.

Main Methods:

• SeCo aggregates high-confidence pixels into semantic regions using Pixel Semantic Aggregation (PSA) and Semantic Connectivity Correction with Loss Distribution (SCC-LD).
• SeCoV2 incorporates SCC-Unc for uncertainty-aware refinement, building a connectivity graph for relational consistency.
• Compatibility with interactive foundation models (SAM, SEEM, Fast-SAM) was validated.

Main Results:

• SeCoV2 achieved consistent improvements across six CDSS tasks, with an average performance gain of up to +4.6%.
• New state-of-the-art results were established in various CDSS benchmarks.
• The framework demonstrated robust adaptation capabilities in diverse, real-world environments.

Conclusions:

• The semantic connectivity-driven approach significantly enhances pseudo-label quality and segmentation accuracy.
• SeCoV2 offers superior generalization and robustness, outperforming previous methods in challenging CDSS scenarios.
• The proposed methods provide a powerful tool for effective domain adaptation in semantic segmentation.