Learn to segment single cells with deep distance estimator and deep cell detector

Single cell segmentation is a critical and challenging step in cell imaging analysis. Traditional processing methods require time and labor to manually fine-tune parameters and lack parameter transferability between different situations. Recently, deep convolutional neural networks (CNN) treat segmentation as a pixel-wise classification problem and have become a general and efficient method for image segmentation. However, cell imaging data often possesses characteristics that adversely affect segmentation accuracy: absence of established training datasets, few pixels on cell boundaries, and ubiquitous blurry features. We developed a strategy that combines strengths of CNN and traditional watershed algorithm. First, we trained a CNN to learn Euclidean distance transform (EDT) of the mask corresponding to the input images (deep distance estimator). Next, we trained a faster R-CNN (Region with CNN) to detect individual cells in the EDT image (deep cell detector). Then, the watershed algorithm performed the final segmentation using the outputs of previous two steps. Tests on a library of fluorescence, phase contrast and differential interference contrast (DIC) images showed that both the combined method and various forms of the pixel-wise classification algorithm achieved similar pixel-wise accuracy. However, the combined method achieved significantly higher cell count accuracy than the pixel-wise classification algorithm did, with the latter performing poorly when separating connected cells, especially those connected by blurry boundaries. This difference is most obvious when applied to noisy images of densely packed cells. Furthermore, both deep distance estimator and deep cell detector converge fast and are easy to train.