A Multimodal Feature Sensing and Fusion Neural Network for Damage Localization by Ultrasonic Guided Waves

Damage localization based on ultrasonic guided waves (UGWs) ensures the reliability and safety of composites. Efficient and accurate damage localization requires full integration of different modality features. However, existing deep learning-based damage localization methods usually focus on single-modal features and cannot deeply mine and fuse different features. In this paper, we propose a novel Multimodal Feature Sensing and Fusion Neural Network (MSFN) for damage localization by UGWs in composites. This method uses an innovative multimodal input mode, in which three different modal signals, namely, the damage signal, scattered wave signal, and energy density signal, are fed into the network as inputs. We use Convolutional Neural Networks (CNNs), Gated Recurrent Units (GRUs) and Bidirectional Gated Recurrent Units (BiGRUs) to construct specific encoders for the characteristics of the three signals to extract the features of different modalities efficiently and quickly. Then we employ an attention mechanism-guided feature fusion strategy to aggregate the various features, map out the correlation between the damage zones and the signal features, and finally decode them through successive linear layers to output the final damage localization results. Subsequent experimental results show that the damage localization accuracy of the MSFN can reach 98.13% even under noise interference. It is shown that its robustness and accuracy are much better than those of other existing networks and it has better localization speed and generalization. The proposed MSFN architecture comprises a CNN-based DS-encoder, a GRU-based SW-encoder, and a BiGRU-based ES-encoder, followed by an attention-guided fusion module, demonstrating its feasibility for near-real-time SHM applications.