Automatic contour quality assurance using deep-learning based contours

Objective.Safe deployment of auto-contouring models requires the inclusion of automated quality assurance (QA). One such approach is to use two independent auto-contouring models and compare them geometrically for acceptability. This is not effective because geometric differences may not correlate with clinically significant errors. Herein, we investigated whether a two-contour QA system is improved by including dose in this comparison.Approach.Volumetric modulated arc therapy plans were generated for 86 head and neck (H&N) and 50 cervical (GYN) cancer patients, using clinically-approved planning target volumes (PTVs) and auto-contour organs-at-risk (OARs) from a primary auto-contouring model. Doses to the primary OARs were compared with doses to manually drawn and approved OARs ('the truth'). A difference inD_meanorD_max⩾ 2 Gy was identified as a reporting error (D_error). A second, independent auto-contouring model was then used to contour the OARs (verification). The primary and verification auto-contouring models were compared geometrically (dice similarity coefficient (DSC), surface DSC, 95% Hausdorff distance, mean surface distance) and dosimetrically (D_mean,D_max). The ability of comparison metrics between the two auto-contouring models to flag actual dosimetric errors (i.e. primary model compared with the truth) was investigated. A logistic regression model was used to predictD_error. The data was divided by disease site and into 50/50 stratified training and testing sets;k-fold cross validation was employed during training to avoid overfitting. H&N structures were further divided into size-specific groups to improve model performance and generalizability.Main results.Including dose metrics in the logistic regression model to predictD_{error, mean}increased the performance in terms of receiver-operating characteristic curve-area under the curve and area under the precision-recall curve in the test set for H&N small structures. ForD_{error, max}, including dose metrics increased performance for H&N small structures, H&N medium structures, and GYN structures.Significance.In many instances, utilizing dose with geometric comparisons can improve the ability of a verification model to flag potential errors from a primary auto-contouring model.