Multi-view attribute learning and context relationship encoding enhanced segmentation of lung tumors from CT images

Graph convolutional neural networks (GCN) have shown the promise in medical image segmentation due to the flexibility of representing diverse range of image regions using graph nodes and propagating knowledge via graph edges. However, existing methods did not fully exploit the various attributes of image nodes and the context relationship among their attributes. We propose a new segmentation method with multi-similarity view enhancement and node attribute context learning (MNSeg). First, multiple views were formed by measuring the similarities among the image nodes, and MNSeg has a GCN based multi-view image node attribute learning (MAL) module to integrate various node attributes learnt from multiple similarity views. Each similarity view contains the specific similarities among all the image nodes, and it was integrated with the node attributes from all the channels to form the enhanced attributes of image nodes. Second, the context relationships among the attributes of image nodes are formulated by a transformer-based context relationship encoding (CRE) strategy to propagate these relationships across all the image nodes. During the transformer-based learning, the relationships were estimated based on the self-attention on all the image nodes, and then they were encoded into the learned node features. Finally, we design an attention at attribute category level (ACA) to discriminate and fuse the learnt diverse information from MAL, CRE, and the original node attributes. ACA identifies the more informative attribute categories by adaptively learn their importance. We validate the performance of MNSeg on a public lung tumor CT dataset and an in-house non-small cell lung cancer (NSCLC) dataset collected from the hospital. The segmentation results show that MNSeg outperformed the compared segmentation methods in terms of spatial overlap and the shape similarities. The ablation studies demonstrated the effectiveness of MAL, CRE, and ACA. The generalization ability of MNSeg was proved by the consistent improved segmentation performances using different 3D segmentation backbones.