CT2X-IRA: CT to x-ray image registration agent using domain-cross multi-scale-stride deep reinforcement learning

Area of Science:

• Medical Imaging
• Computer-Assisted Surgery
• Artificial Intelligence

Background:

• Accurate 3D/2D registration of CT volumes and x-ray images is crucial for computer-assisted minimally invasive surgery.
• Existing registration methods often fail due to initial misalignments and local minima, compromising accuracy and robustness.

Purpose of the Study:

• To develop a novel deep reinforcement learning agent, CT2X-IRA, for robust and accurate 3D/2D CT/x-ray image registration.
• To overcome limitations of previous methods in handling initial misalignments and improving visualization in surgical procedures.

Main Methods:

• Implemented a task-driven deep reinforcement learning framework (CT2X-IRA).
• Incorporated a multi-scale-stride learning mechanism for efficient convergence.
• Utilized a domain adaptation module to bridge the gap between CT and x-ray image domains.
• Employed a weighted reward function to enhance the accuracy of transformation parameter estimation.

Main Results:

• Achieved target registration errors of 2.13 mm and 2.33 mm on public and private clinical datasets.
• Demonstrated fast computation times of 1.5 s and 1.1 s, respectively.
• Validated an accurate and efficient workflow for rigid CT/x-ray image registration.

Conclusions:

• The proposed CT2X-IRA agent provides accurate and robust 3D/2D registration of CT and x-ray images.
• This method shows significant potential for improving intraoperative visualization and guidance in minimally invasive surgery.