Comprehensive Clinical Usability-Oriented Contour Quality Evaluation for Deep Learning Auto-segmentation: Combining...

Area of Science:

• Medical imaging analysis
• Artificial intelligence in healthcare
• Radiology and radiation oncology

Background:

• Current metrics for auto-segmented contour quality have limitations in reflecting clinical usefulness.
• There is a need for improved methods to evaluate the quality of contours generated by deep learning-based auto-segmentation (DLAS).

Purpose of the Study:

• To develop a novel contour quality classification (CQC) method for clinical usability-oriented evaluation of DLAS.
• To combine multiple quantitative metrics into a single classification system.

Main Methods:

• Developed a CQC method using supervised ensemble tree classification models with 7 quantitative metrics.
• Trained organ-specific models for 5 abdominal organs using MRI data.
• Validated models on independent MRI and CT datasets, comparing with interobserver variation (IOV) and a threshold-based approach.

Main Results:

• Achieved high performance with average AUC of 0.982 ± 0.01 (MRI) and 0.979 ± 0.01 (CT).
• Demonstrated high mean accuracy (95.8% ± 1.7% for MRI, 94.3% ± 2.1% for CT) and low risk rate.
• CQC results closely matched IOV and significantly outperformed the threshold-based method.

Conclusions:

• The CQC models exhibit high performance in classifying contour slice quality.
• This method provides an intuitive and comprehensive solution for clinical evaluation of DLAS.
• The CQC addresses limitations of existing metrics, enhancing clinical utility.