DMU-Net: A Dual-Stream Multi-Scale U-Net Network Using Multi-Dimensional Spatial Information for Urban Building...

Area of Science:

• Remote Sensing
• Geographic Information Systems (GIS)
• Computer Vision
• Urban Planning

Background:

• Automated urban building extraction from remote sensing data is crucial for urban planning and management.
• Existing methods often overlook valuable spectral and spatial information beyond standard RGB imagery.
• Gaofen-7 (GF-7) satellite data offers multi-perspective and multispectral capabilities, including three-dimensional spatial information.

Purpose of the Study:

• To develop an advanced deep learning model for accurate urban building extraction using multi-dimensional GF-7 satellite data.
• To leverage Near-Infrared (NIR) and normalized Digital Surface Model (nDSM) data alongside RGB imagery.
• To enhance feature fusion and multi-scale processing for improved extraction performance.

Main Methods:

• A novel dual-stream multi-scale network (DMU-Net) based on U-Net architecture was proposed.
• The encoder utilizes a dual-stream Convolutional Neural Network (CNN) structure, processing RGB, NIR, and nDSM fusion images separately.
• An improved Feature Pyramid Network (IFPN) was integrated into the decoder for effective fusion of multi-band and multi-scale features.

Main Results:

• DMU-Net achieved an Overall Accuracy (OA) of 96.16% and an Intersection-over-Union (IoU) of 84.49% on the GF-7 self-annotated building dataset.
• The inclusion of 3D information (nDSM) significantly boosted extraction accuracy, increasing IoU by 7.61% compared to RGB and 3.19% compared to RGB + NIR.
• DMU-Net demonstrated superior performance over state-of-the-art models like SMU-Net, DU-Net, and IEU-Net, with IoU improvements of 0.74%, 0.55%, and 1.65%, respectively.

Conclusions:

• The proposed DMU-Net effectively utilizes multi-dimensional satellite data for enhanced urban building extraction.
• The dual-stream CNN encoder and IFPN decoder architecture are key to fusing diverse features and improving accuracy.
• Incorporating 3D spatial information is vital for advancing the precision of building extraction tasks in remote sensing.