PMJAF-Net: Pyramidal multi-scale joint attention and adaptive fusion network for explainable skin lesion segmentation

Traditional convolutional neural networks have achieved remarkable success in skin lesion segmentation. However, the successive pooling operations and convolutional spans reduce the feature resolution and hinder the dense prediction for spatial information, resulting in blurred boundaries, low accuracy and poor interpretability for irregular lesion segmentation under low contrast. To solve the above issues, a pyramidal multi-scale joint attention and adaptive fusion network for explainable (PMJAF-Net) skin lesion segmentation is proposed. Firstly, an adaptive spatial attention module is designed to establish the long-term correlation between pixels, enrich the global and local contextual information, and refine the detailed features. Subsequently, an efficient pyramidal multi-scale channel attention module is proposed to capture the multi-scale information and edge features by using the pyramidal module. Meanwhile, a channel attention module is devised to establish the long-term correlation between channels and highlight the most related feature channels to capture the multi-scale key information on each channel. Thereafter, a multi-scale adaptive fusion attention module is put forward to efficiently fuse the scale features at different decoding stages. Finally, a novel hybrid loss function based on region salient features and boundary quality is presented to guide the network to learn from map-level, patch-level and pixel-level and to accurately predict the lesion regions with clear boundaries. In addition, visualizing attention weight maps are utilized to visually enhance the interpretability of our proposed model. Comprehensive experiments are conducted on four public skin lesion datasets, and the results demonstrate that the proposed network outperforms the state-of-the-art methods, with the segmentation assessment evaluation metrics Dice, JI, and ACC improved to 92.65%, 87.86% and 96.26%, respectively.