Synthesis of fracture radiographs with deep neural networks

We describe a machine learning system for converting diagrams of fractures into realistic X-ray images. We further present a method for iterative, human-guided refinement of the generated images and show that the resulting synthetic images can be used during training to increase the accuracy of deep classifiers on clinically meaningful subsets of fracture X-rays.

A neural network was trained to reconstruct images from programmatically created line drawings of those images. The images were then further refined with an optimization-based technique. Ten physicians were recruited into a study to assess the realism of synthetic radiographs created by the neural network. They were presented with mixed sets of real and synthetic images and asked to identify which images were synthetic. Two classifiers were trained to detect humeral shaft fractures: one only on true fracture images, and one on both true and synthetic images.

Physicians were 49.63% accurate in identifying whether images were synthetic or real. This is close to what would be expected by pure chance (i.e. random guessing). A classifier trained only on real images detected fractures with 67.21% sensitivity when no fracture fixation hardware was present. A classifier trained on both real images and synthetic images was 75.54% sensitive.

Our method generates X-rays realistic enough to be indistinguishable from real X-rays. We also show that synthetic images generated using this method can be used to increase the accuracy of deep classifiers on clinically meaningful subsets of fracture X-rays.