UAV Imagery Real-Time Semantic Segmentation with Global-Local Information Attention

Area of Science:

• Computer Vision
• Artificial Intelligence
• Remote Sensing

Background:

• Lightweight algorithms for real-time semantic segmentation in drone imagery often fail to integrate global and local image information effectively.
• This deficiency leads to missed detections and misclassifications, hindering performance in critical applications.

Purpose of the Study:

• To propose a novel method for real-time semantic segmentation of drone imagery that enhances the integration of multi-scale global context information.
• To improve accuracy and real-time processing capabilities compared to existing lightweight models.

Main Methods:

• Utilized a UNet structure with a Resnet18 encoder for feature extraction.
• Incorporated a global-local attention module in the decoder to fuse global and local image information.
• Employed a shallow-feature fusion module in the segmentation head for multi-scale feature integration.

Main Results:

• Achieved 68% mIoU on the UAvid dataset and 67% mIoU on the UDD6 dataset, surpassing the baseline UNet by 10% and 21.2%, respectively.
• Reached a real-time processing speed of 72.4 frames/s, which is 54.4 frames/s faster than the baseline UNet.
• Demonstrated a well-balanced performance between accuracy and real-time processing speed.

Conclusions:

• The proposed method effectively integrates multi-scale global context and local information for enhanced real-time semantic segmentation.
• The model significantly improves accuracy and achieves high processing speeds, making it suitable for demanding drone imagery applications.