CGDFNet: a dual-branch real-time semantic segmentation network with context-guided detail fusion

Area of Science:

• Computer Vision
• Deep Learning
• Image Segmentation

Background:

• Dual-branch networks are vital for real-time semantic segmentation.
• Existing methods suffer from detail loss during downsampling and underutilize contextual information.
• Traditional fusion methods struggle to integrate local and global features effectively.

Purpose of the Study:

• To develop a Context-Guided Detail Fusion Network (CGDFNet) for enhanced feature representation and detail preservation in semantic segmentation.
• To address limitations in existing dual-branch networks regarding detail loss and contextual information underutilization.

Main Methods:

• Implemented a Semantic Refinement Module (SRM) in the context branch for adaptive pooling and parallel feature processing.
• Introduced a Context-Guided Detail Module (CGDM) in the detail branch to reinforce high-frequency features using semantic information.
• Developed a Fourier-Domain Adaptive Fusion Module (FDAFM) for efficient fusion of dual-branch features via adaptive gating and Fourier transform.

Main Results:

• CGDFNet achieved 77.8% mIoU at 87.6 FPS on the Cityscapes test set.
• CGDFNet attained 77.9% mIoU at 128.7 FPS on the CamVid test set.
• Experimental results demonstrate a balance between segmentation quality and real-time inference speed.

Conclusions:

• CGDFNet effectively enhances feature representation by preserving image details and leveraging contextual information.
• The proposed network achieves state-of-the-art performance in real-time semantic segmentation.
• CGDFNet offers a promising solution for applications requiring both accuracy and speed in semantic segmentation.