Predicting the Effort Required to Manually Mend Auto-Segmentations.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Radiation Oncology

Background:

• Auto-segmentation accuracy is crucial for clinical utility.
• Existing metrics like Dice Coefficient (DC) and Hausdorff Distance (HD) fail to capture manual correction effort.
• Clinical efficiency requires evaluating the time experts spend mending auto-segmentations.

Purpose of the Study:

• To explore methods for evaluating auto-segmentations considering clinical efficiency.
• To assess explicit metrics and novel deep learning approaches for predicting manual mending effort.
• To validate these methods across multiple institutions and organs for radiation therapy planning.

Main Methods:

• Recorded expert correction time to establish ground-truth mending effort.
• Evaluated five explicit metrics, including Mendability Index (MIhd) and spatial Hausdorff Distance (sHD).
• Developed and tested deep learning networks to implicitly predict mending effort using auto-segmentations and original images.

Main Results:

• MIhd best predicted mending effort for sparse objects (6.2-14.4% error).
• sHD performed best for large, non-sparse objects.
• Deep learning models accurately predicted mending effort (2.9-12.9% error) without needing ground-truth segmentations.

Conclusions:

• Explicit metrics have limitations in predicting clinical mending effort.
• Deep learning offers a promising approach to predict mending effort implicitly and efficiently.
• Effort-predicting deep models can assess clinical usability beyond technical evaluations.