A Retinal Image Sequence Registration Method Based on Longitudinal 3D Fundoscopy Scene Modeling

Accurate and reliable global registration is the basis and prerequisite for longitudinal studies of related diseases that are based on retinal image sequences. However, most existing retinal image registration methods do not consider the implicit inherent spatiotemporal transformation process between the retinal images to be registered, resulting in low accuracy or poor reliability. In this paper, we propose a 3D spatiotemporal model that simulates the fundoscopy scene which is on a longitudinal time-axis. There are two main components: 1) a dynamic eyeball model that simulates the changes in eyeball shape owing to natural growth or disease progression (e.g., myopia); and 2) a camera array model that simulates the changes in eyeball pose and fundus camera parameters between each fundoscopy. Based on the 3D spatiotemporal model, we implement a retinal image sequence registration framework using a frame-to-reference and frame-to-frame joint registration strategy. The framework uses the matched keypoints as a medium, and adopts pose estimation and optimization algorithm to align all images in the sequence into the same longitudinal fundoscopy scene. Benefiting from the spatiotemporal simulation of the longitudinal fundoscopy scene, the proposed method can generate a large amount of realistic synthetic data given only one retinal image. We conduct comprehensive experiments on two image pair registration datasets and three image sequence registration datasets. The results show that the proposed method achieves state-of-the-art registration accuracy, reliability, and applicability. To promote the study in related fields, we make all codes and datasets publicly available.