Development and validation of a quality control procedure for automatic segmentation of hippocampal subfields

Automatic segmentation methods for in vivo magnetic resonance imaging are increasing in popularity because of their high efficiency and reproducibility. However, automatic methods can be perfectly reliable and consistently wrong, and the validity of automatic segmentation methods cannot be taken for granted. Quality control (QC) by trained and reliable human raters is necessary to ensure the validity of automatic measurements. Yet QC practices for applied neuroimaging research are underdeveloped. We report a detailed QC and correction procedure to accompany our validated atlas for hippocampal subfield segmentation. We document a two-step QC procedure for identifying segmentation errors, along with a taxonomy of errors and an error severity rating scale. This detailed procedure has high between-rater reliability for error identification and manual correction. The latter introduces at maximum 3% error variance in volume measurement. All procedures were cross-validated on an independent sample collected at a second site with different imaging parameters. The analysis of error frequency revealed no evidence of bias. An independent rater with a third sample replicated procedures with high within-rater reliability for error identification and correction. We provide recommendations for implementing the described method along with hypothesis testing strategies. In sum, we present a detailed QC procedure that is optimized for efficiency while prioritizing measurement validity and suits any automatic atlas.