Skeleton extraction and inpainting from poor, broken ESPI fringe with an M-net convolutional neural network

Extracting skeletons from fringe patterns is the key to the fringe skeleton method, which is used to extract phase terms in electronic speckle pattern interferometry (ESPI). Because of massive inherent speckle noise, extracting skeletons from poor, broken ESPI fringe patterns is challenging. In this paper, we propose a method based on a modified M-net convolutional neural network for skeleton extraction from poor, broken ESPI fringe patterns. In our method, we pose the problem as a segmentation task. The M-net performs excellent segmentation, and we modify its loss function to suit our task. The broken ESPI fringe patterns and corresponding complete skeleton images are used to train the modified M-net. The trained network can extract and inpaint the skeletons simultaneously. We evaluate the performance of the network on two groups of computer-simulated ESPI fringe patterns and two groups of experimentally obtained ESPI fringe patterns. Two related recent methods, the gradient vector fields based on variational image decomposition and the U-net based method, are compared with our method. The results demonstrate that our method can obtain accurate, complete, and smooth skeletons in all cases, even where fringes are broken. It outperforms the two compared methods quantitatively and qualitatively.