Multimodal Building Damage Assessment Method Fusing Adaptive Attention Mechanism and State-Space Modeling

Area of Science:

• Remote Sensing
• Artificial Intelligence
• Disaster Management

Background:

• Accurate building damage assessment (BDA) is vital for effective post-disaster response.
• Current BDA methods struggle with background noise, limited spatial dependency modeling, and adverse weather.
• The need for robust BDA solutions that integrate multi-modal data is critical.

Purpose of the Study:

• To propose the Adaptive Difference State-Space Fusion Network (ADSFNet) for enhanced BDA.
• To improve resilience against weather-induced limitations by processing both optical and Synthetic Aperture Radar (SAR) data.
• To develop an intelligent emergency decision-making system using disaster-centric knowledge graphs and Large Language Models.

Main Methods:

• ADSFNet utilizes an Adaptive Difference Attention Fusion (ADAF) module to focus on building damage while reducing background interference.
• The Hybrid Selective State-Space Convolution (HSSC) module combines Convolutional Neural Networks (CNNs) for local texture and Mamba for global spatial dependencies.
• The network processes fused optical and SAR imagery for comprehensive damage analysis.

Main Results:

• ADSFNet achieved superior performance on the BRIGHT and xBD datasets, with F1 scores of 71.36% and 73.98%, respectively.
• The proposed method demonstrated improvements of 1.29% and 0.6% over state-of-the-art techniques.
• The model successfully integrated with Large Language Models for an intelligent disaster management system.

Conclusions:

• ADSFNet offers a significant advancement in building damage assessment by effectively fusing multi-modal data.
• The network's ability to model both local details and global context enhances accuracy under challenging conditions.
• The developed intelligent management system provides a novel pathway for improved emergency response and decision-making.