A hybrid attention multi-scale fusion network for real-time semantic segmentation

Area of Science:

• Computer Vision
• Deep Learning
• Image Segmentation

Background:

• Semantic segmentation algorithms often prioritize semantic information over spatial details, leading to accuracy loss.
• Real-time inference speed improvements in current methods come at the cost of reduced spatial information retrieval.

Purpose of the Study:

• To propose a new semantic segmentation method that effectively retrieves lost spatial information.
• To enhance object classification accuracy and boundary information extraction.
• To improve the balance between inference speed and accuracy in semantic segmentation.

Main Methods:

• Designed a Hierarchical Feature Fusion Module (HFFM) to merge multi-level features and capture larger receptive fields using attention.
• Developed a Hierarchical Feature Retrieval Module (HFRM) combining channel and spatial attention to recover downsampling-induced spatial information loss.
• Integrated edge detection techniques to refine boundary information extraction.

Main Results:

• Achieved 73.6% mIoU at 176 FPS on Cityscapes (512x1024 input).
• Achieved 70.0% mIoU at 146 FPS on Camvid (512x1024 input).
• Demonstrated superior performance compared to existing networks in terms of speed and accuracy.

Conclusions:

• The proposed method effectively recovers spatial information, improving semantic segmentation accuracy.
• The integration of HFRM and HFFM modules offers a practical solution for real-time semantic segmentation with high accuracy.
• The approach enhances the practicality of semantic segmentation models for real-world applications.