Patch-sampled contrastive learning for dense prediction pretraining in metallographic images

The shapes, sizes, and distributions of microstructures determine the mechanical properties of alloy products, necessitating identification of microstructures in metallographic images during the manufacturing process. Previous studies demonstrated the success of deep learning in analyzing such images. However, existing methods do not fully consider the high cost of annotations, which limits the application of deep learning-based methods. Considering the popularity of self-supervised learning for natural images, further research is required to design a microstructure-specific pretraining framework. To this end, we propose a novel patch-sampled contrastive learning (PSCL) method, considering the characteristics of metallographic image segmentation tasks. First, image- and patch-level contrastive learning frameworks are designed to help the model effectively capture visual features. The former captures global features, whereas the latter captures local features in greater detail. Second, a multiscale strategy is introduced to improve the adaptability of the subsequent microstructure segmentation. Finally, considering that patches may concurrently contain multiple microstructures, a sampling method based on feature similarity was designed to capture the discriminable features of different categories. In experiments, after model fine-tuning of only one annotated image, the proposed method achieved a Dice score of 0.6296 which is higher than those of existing self-supervised learning methods with identical model structure, proving the effectiveness of the method in microstructure segmentation. The source code and dataset of the proposed PSCL in this work are available at https://github.com/neulmc/PSCL.