RMSim: controlled respiratory motion simulation on static patient scans

Area of Science:

• Medical Imaging
• Deep Learning
• Computational Anatomy

Background:

• Deformable image registration (DIR) is crucial for medical image analysis.
• Accurate validation and training data are needed for DIR algorithms, especially deep learning-based methods.
• Simulating realistic anatomical deformations, particularly respiratory motion, is challenging.

Purpose of the Study:

• To develop a deep learning-based respiratory motion simulator (RMSim) to generate realistic anatomical deformations from static patient scans.
• To provide ground truth data for validating DIR algorithms.
• To improve deep learning-based DIR algorithms through data augmentation.

Main Methods:

• A novel 3D Seq2Seq deep learning model (RMSim) was developed using stacked 3D-ConvLSTMs.
• RMSim learns from 4D-CT images to predict breathing phases and generates time-varying displacement vector fields (DVFs).
• The model incorporates auxiliary 1D breathing traces and includes smoothness loss and mean-squared error in its training objective.

Main Results:

• RMSim achieved a structure similarity index measure (SSIM) of 0.92 ± 0.04 when compared to ground truth 4D CT images, indicating realistic motion simulation.
• The use of RMSim-augmented data improved landmark registration error in a public DIR dataset from 8.12 ± 5.78 mm to 6.58 ± 6.38 mm.
• The simulator was validated on healthy and cancer patient datasets.

Conclusions:

• The proposed RMSim effectively generates realistic respiratory motion and anatomical deformations.
• RMSim serves as a valuable tool for validating DIR algorithms and augmenting training datasets for deep learning-based DIR.
• The generated data can lead to more accurate and robust DIR algorithms in clinical practice.