Automatic segmentation of the pectoral muscle based on boundary identification and shape prediction

The purpose of this work is to identify the pectoral muscle region in mediolateral oblique (MLO) view mammograms even when the boundary is blurred or obscured. The problem is decoupled into two subproblems in our study: identifying parts of boundaries with high confidence and predicting the overall shape of the pectoral muscle. Due to the similarity in intensity and texture between pectoral muscle and gland tissue, we trained a deep neural network to distinguish them in the first subproblem. The boundary with high confidence can be obtained according to the consistency of predictions from multiple converged models. For the shape prediction problem, a generative adversarial network (GAN) is used to learn mapping from a given identified region and the breast shape to the overall pectoral muscle shape. Our method is evaluated on a mammogram dataset including 633 MLO view mammograms collected from three different datacenters. We take U-Net as our baseline model and the dataset is divided into three groups according to the performance of U-Net for evaluation. In all three groups, U-Net achieves 80.1%, 92.9%, and 98.3% in the Dice similarity coefficient, respectively, and our method achieves 85.2%, 94.8%, and 98.1% in the Dice similarity coefficient, respectively. The experiment shows that our method effectively estimates the pectoral muscle boundary, even parts of boundaries that are difficult to detect, and greatly improves the performance of segmentation in this case.