A Dual-Branch Network with Mixed and Self-Supervision for Medical Image Segmentation: An Application to Segment...

In clinical applications, one often encounters reduced segmentation accuracy when processing out-of-distribution (OOD) patient data. Segmentation models could be leveraged by utilizing either transfer learning or semi-supervised learning on a limited number of strong labels from manual annotation. However, over-fitting could potentially arise due to the small data size. This work develops a dual-branch network to improve segmentation on OOD data by also applying a large number of weak labels from inaccurate results generated by existing segmentation models. The dual-branch network consists of a shared encoder and two decoders to process strong and weak labels, respectively. Mixed supervision from both labels not only transfers the guidance from the strong decoder to the weak one, but also stabilizes the strong decoder. Additionally, weak labels are iteratively replaced with the segmentation masks from the strong decoder by self-supervision. We illustrate the proposed method on the adipose tissue segmentation of 40 patients with edema. Image data from edematous patients are OOD for existing segmentation methods, which often induces under-segmentation. Overall, the dual-branch segmentation network yielded higher accuracy than two baseline methods; the intersection over union (IoU) improved from 60.1% to 71.2% (p < 0.05). These findings demonstrate the potential of the dual-branch segmentation network with mixed- and self-supervision to process the OOD data in clinical applications.