Combining multimodal fatigue fracture surface images for analysis with a CNN

This work uses three different modalities, namely SEM, BSE and scanning white light interference (SWLI) to image fatigue fracture surfaces of Ti-6Al-4V. Convolutional neural networks (CNNs) that were pre-trained on images of the natural world were used to predict values such as distance from load line and crack growth rate. SEM images are routinely used to study the topography of fracture surfaces because the shallower interaction volume resolves surface features while BSE images and SWLI data add information about composition and surface height. Combining the three imaging modalities via the use of color channels facilitates overlaying them on the same grid for transferability of models pre-trained on colored images. This work shows that the imaging modalities under the guise of color channels have different levels of importance depending on the model being trained. It also documents how the combination of information from these modalities improves the classification and regression results by 20 and 60 %, respectively, relative to the secondary electron images alone. (162/200).