Application of optical flow algorithm for drift correction in electron microscopy images

Transmission electron microscopy (TEM) image drift correction has been effectively addressed using diverse approaches, including the cross correlation algorithm (CC) and other strategies. However, most of the strategies fall short of achieving sufficient accuracy or cannot strike a balance between time consumption and accuracy. The present study proposes a TEM image drift correction strategy that enhances accuracy without any additional time consumption. Unlike the CC algorithm that matches pixels one by one, our approach involves the extraction of multiple feature points from the first TEM image and then uses the Lucas-Kanade (LK) optical flow algorithm to calculate the optical field of these feature points in the subsequent TEM images. The LK algorithm is used to calculate the instantaneous velocity of these feature points, which can help track the movement of the TEM image series. In addition, a high-precision sub-pixel level correction strategy by the utilization of linear interpolation during the correction process is developed in this work. Experimental results confirm that this strategy offers superior accuracy in comparison with the CC algorithm and also is insensitive to the size of the image. Furthermore, we offer a semantic segmentation neural network for electron microscope image pre-processing, thereby expanding the applicability of our methodology.