Evaluation of Projection Images for Visual Quality Control of Automated Left and Right Lung Segmentations on...

Background/Objectives: To assess diagnostic accuracy of two-dimensional (2D) projection methods for rapid visual quality control of automated volumetric (3D) lung segmentations compared with slice-based 3D review of segmentation results for application in large-scale studies. Methods: Segmentation of right and left lungs on T1-weighted MRI of 300 participants of the German National Cohort (NAKO) study was performed using the nnU-NET framework. Three variants of 2D projection images of segmentation masks were created: maximum intensity projection (MIP) using pseudo-chromadepth encoding with different color spectra for right and left lung (Colored_MIP) and standard deviation projection of segmentation mask outlines, encoded in black-and-white (Gray_outline) or using color-encoding (Colored_outline). The worst of two ratings by two independent raters conducting slice-based review for segmentation errors on underlying imaging data and review for mislabeling errors served as the standard of reference. All variants were evaluated by five raters each for identification of segmentation errors and the majority rating was used as index test. The time required for review was recorded and diagnostic accuracies were calculated. Results: Sensitivities of Colored_MIP, Colored_outline and Gray_outline were 88.2% [95%-CI 78.7%; 94.4%], 89.5% [80.3%; 95.3%] and 78.9% [68.1%; 87.5%]; specificities were 98.7% [96.1%; 99.7%], 96.4% [93.1%; 98.5%] and 98.7% [96.1%; 99.7%]; and F1-scores were 0.918, 0.895 and 0.863, respectively. Mean time per case and rater required for evaluation was 2.8 ± 0.9 s for Colored_outline, 1.7 ± 0.1 s for Colored_MIP, and 2.0 ± 0.4 s for Gray_outline. Conclusions: The 2D segmentation mask projection images enabled the detection of segmentation errors of automated 3D segmentations of left and right lungs based on MRI with high diagnostic accuracy, especially when using color-encoding. The method enabled evaluation within a matter of seconds per case. Segmentation mask projection images may assist in visual quality control of automated segmentations in large-scale studies.