Analysis of manual segmentation in paranasal CT images

Manual segmentation is often used for evaluation of automatic or semi-automatic segmentation. The purpose of this paper is to describe the inter and intraindividual variability, the dubiety of manual segmentation as a gold standard and to find reasons for the discrepancy. We realized two experiments. In the first one ten ENT surgeons, ten medical students and one engineer outlined the right maxillary sinus and ethmoid sinuses manually on a standard CT dataset of a human head. In the second experiment two participants outlined maxillary sinus and ethmoid sinuses five times consecutively. Manual segmentation was accomplished with custom software using a line segmentation tool. The first experiment shows the interindividual variability of manual segmentation which is higher for ethmoidal sinuses than for maxillary sinuses. The variability can be caused by the level of experience, different interpretation of the CT data or different levels of accuracy. The second experiment shows intraindividual variability which is lower than interindividual variability. Most variances in both experiments appear during segmentation of ethmoidal sinuses and outlining hiatus semilunaris. Concerning the inter and intraindividual variances the segmentation result of one manual segmenter could not directly be used as gold standard for the evaluation of automatic segmentation algorithms.