Asymmetric Network Combining CNN and Transformer for Building Extraction from Remote Sensing Images

Area of Science:

• Computer Vision
• Remote Sensing
• Artificial Intelligence

Background:

• Accurate building extraction from remote sensing data is vital for urban planning, disaster response, and defense.
• Transformer models excel at global context but are computationally intensive and lack local detail.
• Convolutional Neural Networks (CNNs) are adept at local feature extraction but struggle with global information.

Purpose of the Study:

• To propose an efficient and accurate building extraction model by synergizing CNN and Transformer architectures.
• To address the limitations of existing models, namely high computational cost and insufficient local or global information processing.

Main Methods:

• Introduced CTANet, an asymmetric network combining ConvNeXt encoder for feature extraction and a bilateral hybrid attention transformer (BHAFormer) decoder.
• BHAFormer leverages texture edge and background information for global dependency modeling, enhancing accuracy and efficiency.
• Incorporated a multiscale mixed attention mechanism module (MSM-AMM) to refine features, reduce noise, and compensate for downsampling information loss.

Main Results:

• CTANet achieved superior performance on three benchmark datasets: Massachusetts, WHU, and Inria.
• The model demonstrated the highest F1-scores (86.7%, 95.74%, 90.52%) and IoU (76.52%, 91.84%, 82.68%) compared to state-of-the-art methods.
• The proposed architecture effectively balances global and local feature extraction for precise building detection.

Conclusions:

• CTANet offers an efficient and effective solution for building extraction from remote sensing images.
• The integration of CNN and Transformer components, along with novel attention mechanisms, significantly advances the field.
• The model's strong performance highlights its potential for real-world applications in urban management and emergency response.