Circular shape constrained fuzzy clustering (CiscFC) for nucleus segmentation in Pap smear images

Accurate detection and segmentation of cell nucleus is the precursor step towards computer aided analysis of Pap smear images. This is a challenging and complex task due to degree of overlap, inconsistent staining and poor contrast. In this paper, a novel nucleus segmentation method is proposed by incorporating a circular shape function in fuzzy clustering. The proposed method was evaluated quantitatively and qualitatively using the Overlapping Cervical Cytology Image Segmentation Challenge - ISBI 2014 challenge dataset comprised of 945 overlapping Pap smear images. It achieved superior performance in terms of Dice similarity coefficient of 0.938, pixel-based recall 0.939 and object based precision 0.968. The results were compared with the standard fuzzy c-means (FCM) clustering, ISBI 2014 challenge submissions and recent state-of-the-art methods. The outcome shows that the new approach can produce more accurate nucleus boundaries while keeping high level of precision and recall.