Quantifying manual editing time in autocontouring via audit log analysis

Artificial intelligence (AI)-based contouring reduces delineation workload, yet manual corrections remain required. Reliable, scalable quantification of structure-level editing time is needed to evaluate clinical usability beyond geometric similarity metrics as geometric metrics may not reflect how tooling impacts edit time.

This retrospective observational study analysed 3083 AI-generated, edited structures created in routine practice. Structure-modifying actions were extracted from the treatment planning system audit log database and converted to per-structure editing time by summing inter-event intervals. Automatic times were benchmarked against manual time recordings for 54 structures across 8 patient cases using Lin's concordance correlation coefficient (CCC). Added path length (APL) between AI and corrected contours was computed, and the associations between editing time, APL and tooling (interpolation use) were investigated using Spearman's correlation coefficient.

Audit log-derived editing time agreed well with manual timing (CCC = 0.93). In the full cohort, 3083/24970 structures were edited (12% correction rate). Brush tools were used in 80% and contour interpolation in 42% of edited structures. Median editing time and APL were higher with interpolation than without (81 s vs 58 s; 232 mm vs 18 mm; both p < 0.05), and correlation between editing time and APL was fair (r = 0.38 vs 0.51).

Vendor audit logs could be translated into accurate structure-level editing time estimates, enabling low-burden monitoring of manual workload. Editing strategy influenced time-geometry relationships, supporting log-based time endpoints alongside geometric metrics for meaningful auto-contouring evaluation.