Implementation of an automated contour quality assurance tool within the TROG 18.01 NINJA trial

Automated contour quality assurance (QA) has the potential to reduce resource and cost requirements for clinical trial QA. While several studies have developed such models, few have been translated for use in prospective real-world trials. This study aimed to implement a contour QA tool using a previously developed model and evaluate its performance during deployment within the TROG 18.01 NINJA prostate cancer trial.

A software tool was developed using an existing prostate clinical target volume (CTV) QA model and integrated into the trial QA workflow. A pilot study was conducted over an 18-month period, during which 56 CTVs were assessed. Reports generated by the tool flagged cases for review and were provided to radiation oncologists to support QA processes.

All five protocol-violating CTVs were correctly identified during deployment, yielding a sensitivity of 1.0. However, a higher-than-expected false positive rate resulted in an accuracy of 0.46 and specificity of 0.41. Retrospective analysis showed that many cases submitted deviated from the model's training distribution, primarily due to inconsistencies in MRI acquisition and variation in submitted CTV definitions. Incorporating out-of-distribution detection based on histogram correlation and model uncertainty improved accuracy to 0.69 for in-distribution cases. Radiation oncologists reported time savings of up to 60 min per case. However, preparation of data remained time intensive for QA coordinators, highlighting the need for further workflow automation.

These findings support the feasibility of automated contour QA in multicentre trials and offer guidance for future implementation at scale.