Where's the naevus? Inter-operator variability in the localization of melanocytic lesion border

The first step in the analysis of a dermatoscopically imaged melanocytic lesion is segmentation--informally, isolating those points in the image belonging to the lesion from those belonging to the surrounding non-lesional skin. Although typically studied in the context of automated analysis, segmentation is a necessary step even for human operators who plan to evaluate quantitative features of a lesion (such as diameter or asymmetry).

In a double blind evaluation of the segmentation of 77 digital dermatoscopic images, we observed a significant inter-operator variability.

The area of the disagreement region was on average 15.28% of the area of the lesion itself, and in 10% of the cases it was more than 28%. More experienced dermatologists showed greater agreement among themselves than with less experienced dermatologists, and a slight tendency toward 'tighter' segmentations.

The evaluation methodology addresses a number of crucial difficulties encountered in previous studies and may be of independent interest. Our results underscore the necessity of taking into account inter-operator variability in large epidemiological studies, in particular those involving less experienced dermatologists, and of striving toward techniques allowing greater standardization and replicability in the evaluation of the fundamental visual parameters of lesions.