A novel DLDRM: Deep learning-based flood disaster risk management framework by multimodal social media data

Area of Science:

• Computer Science
• Artificial Intelligence
• Disaster Management

Background:

• Social media is crucial for disaster information dissemination.
• Existing research often uses unimodal data (text or images) for disaster classification.
• A gap exists in effectively integrating multimodal social media data for disaster response.

Purpose of the Study:

• To develop a multimodal deep learning approach integrating text and visual data from disaster-related social media posts.
• To introduce a novel DL-based disaster risk management (DLDRM) structure for classifying multimodal disaster data.
• To evaluate the performance of DLDRM against established multimodal models.

Main Methods:

• A multimodal deep learning approach was developed by combining text and image data from Twitter posts during disasters.
• A novel DL-based disaster risk management (DLDRM) structure was proposed for multimodal disaster data classification.
• DLDRM was compared against VGG 16, VGG 19, ResNet 50, DenseNet 121, and RegNet Y320 using benchmark datasets.

Main Results:

• The proposed DLDRM model achieved high performance metrics: 99% accuracy, 92.5% precision, 84.08% recall, and 98.5% F1-score.
• DLDRM demonstrated superior performance compared to existing state-of-the-art fusion techniques on a benchmark multimodal disaster dataset.
• The model effectively emphasizes pertinent aspects of both text and image tweets for improved classification.

Conclusions:

• The developed multimodal deep learning technique offers a significant advancement in disaster information classification.
• DLDRM provides an effective framework for disaster risk management by leveraging integrated social media data.
• This approach surpasses current methods, highlighting the potential of multimodal analysis in disaster response.