Evaluation of model uncertainty in AI-based synthetic CT generation from CBCT for abdominal adaptive radiotherapy
Paulo Quintero1, Laura Cerviño1, Hao Zhang1
1Medical Physics Department, Memorial Sloan Kettering Cancer Center, New York, USA.
Background:
The synthesis of CT from CBCT images using AI methods has been explored in radiotherapy to improve adaptive workflows. However, the model training process can be particularly challenging for the abdominal region due to dataset disparities between CT and CBCT images caused by organ motion, low soft tissue contrast, and inconsistencies in air volumes. These factors might impact the implicit prediction uncertainties, which are not actively considered on the synthetized images, overlooking poorly predicted image regions that might lead to inaccuracies in the dose calculation.
Purpose:
To evaluate the impact of the model uncertainty on the predicted Hounsfield Units (HU) and dose calculation on synthetic CT (sCT) for abdominal patients.
Methods:
CBCT images from 65 abdominal patients were retrospectively used to generate sCT images. Rigid image registration (RIR) and deformable image registration (DIR) were individually implemented to create two datasets (D1 and D2) to train (80%), validate (10%), and test (10%) three models (M1: Unet, M2: Bayes-Unet, M3: cycle-GAN). Treatment plans were made on the ground truth CT (GTCT) and the sCTs for dose calculation comparison. The model performance was evaluated with mean absolute error (MAE) and root mean square error (RMSE), and the sCT quality was verified with structural similarity index measure (SSIM). Gamma index (2%/2 mm), D95% of PTV, and Dmean of liver were evaluated and compared between the plans calculated on the GTCT and the sCT. The voxel-wise uncertainty map for M1 and M3 were generated by calculating the standard variation of each voxel from training the model independently ten times. For M2 the Monte Carlo DropConnect method was implemented with 100 iterations. Finally, the uncertainty was associated with the accuracy of CT numbers and dose calculation.
Results:
Across the three models {M1, M2, M3} trained with D1 and D2, the MAE were {50.9 ± 13.3} and {40.9 ± 11.5}, respectively, the RMSE were {68.3 ± 13.5} and {62.2 ± 10.7}, respectively, and the SSIM were {0.89 ± 0.05} and {0.94 ± 0.05}, respectively. For D1 and D2, the gamma rates were {96.3 ± 1.04} and {97.3 ± 0.2}, respectively. No major differences in DVH were noticed between GTCT and sCT (p < 00.1). The correlation between the whole sCT uncertainty maps and gamma index was statistically significant (Spearman's coefficient = 0.84, p < 0.001) and weak between the target volume uncertainty and gamma index (Spearman's coefficient = 0.01, p = 0.89).
Conclusion:
Using DIR resulted in improved performance across all three models. Metrics used to evaluate synthetic image accuracy might not reflect the uncertainty implications in image quality and dose calculations, which suggests the benefit of displaying uncertainty errors in AI generated sCT as a potential strategy to improve the evaluation of intra-fraction changes used for adaptive abdominal radiotherapy.
