Discussion of a Simple Method to Generate Descriptive Images Using Predictive ResNet Model Weights and Feature Maps...

Predictive models like Residual Neural Networks (ResNets) can use Magnetic Resonance Imaging (MRI) data to identify cervix tumors likely to recur after radiotherapy (RT) with high accuracy. However, there persists a lack of insight into model selections (explainability). In this study, we explored whether model features could be used to generate simulated images as a method of model explainability.

T2W MRI data were collected for twenty-seven women with cervix cancer who received RT from the TCGA-CESC database. Simulated images were generated as follows: [A] a ResNet model was trained to identify recurrent cervix cancer; [B] a model was evaluated on T2W MRI data for subjects to obtain corresponding feature maps; [C] most important feature maps were determined for each image; [D] feature maps were combined across all images to generate a simulated image; [E] the final image was reviewed by a radiation oncologist and an initial algorithm to identify the likelihood of recurrence.

Predictive feature maps from the ResNet model (93% accuracy) were used to generate simulated images. Simulated images passed through the model were identified as recurrent and non-recurrent cervix tumors after radiotherapy. A radiation oncologist identified the simulated images as cervix tumors with characteristics of aggressive Cervical Cancer. These images also contained multiple MRI features not considered clinically relevant.

This simple method was able to generate simulated MRI data that mimicked recurrent and non-recurrent cervix cancer tumor images. These generated images could be useful for evaluating the explainability of predictive models and to assist radiologists with the identification of features likely to predict disease course.