Enabling machine learning in X-ray-based procedures via realistic simulation of image formation

Area of Science:

• Medical Imaging
• Radiology
• Machine Learning

Background:

• Deep learning excels in radiology but has lagged in image-guided procedures due to data limitations.
• Archiving procedural images and annotation are significant challenges for deep learning model development.

Purpose of the Study:

• To develop and evaluate a framework for simulating X-ray images from CT data for machine learning applications.
• To assess the generalization capabilities of models trained on simulated fluoroscopy images for image-guided procedures.

Main Methods:

• Extended the DeepDRR framework for realistic X-ray simulation with tool modeling capabilities.
• Integrated DeepDRR with Python, PyTorch, and PyCuda for machine learning.
• Trained convolutional neural networks (ConvNets) on simulated (DeepDRRs) and non-simulated (naïve DRRs) data for anatomical landmark detection and robotic tool localization.

Main Results:

• ConvNets trained on DeepDRRs significantly outperformed those trained on naïve DRRs when tested on real X-ray data.
• Performance was consistent across anatomical landmark detection and robotic tool segmentation tasks.

Conclusions:

• The DeepDRR framework facilitates the use of machine learning in X-ray-guided procedures by enabling training on simulated data.
• This approach shows promise for revolutionizing intra-operative image analysis and simplifying surgical workflows.