Large scale simulation of labeled intraoperative scenes in unity.

Area of Science:

• Medical imaging
• Computer-assisted surgery
• Machine learning

Background:

• Access to large, diverse datasets is crucial for training robust medical imaging models.
• Real-world data for surgical training is often limited due to privacy and availability constraints.

Purpose of the Study:

• To investigate the efficacy of synthetic data generated via the Unity game engine for liver segmentation in laparoscopic surgery.
• To compare the performance of models trained on synthetic data versus real-world data.

Main Methods:

• Utilized the Unity game engine and Perception package to simulate complex intraoperative scenes and generate automatically labeled data.
• Trained a U-Net model using 50,000 simulated liver images without manual annotation.
• Compared the synthetic data-trained model against a model trained on 950 manually labeled laparoscopic images.

Main Results:

• Synthetic data models achieved an average DICE score of 0.59, while real data models achieved 0.64.
• A combined approach using both synthetic and real data yielded the highest average DICE score of 0.75.
• Synthetic data models demonstrated the ability to generalize to real patient data, outperforming real data models in 40% of cases.

Conclusions:

• Synthetic data generated with this method can be effectively used for training medical imaging models, showing comparable performance to real data.
• Pre-training models with synthetic data enhances performance when subsequently trained on real-world data.
• This approach offers a scalable solution to data scarcity in medical applications like image-guided surgery.