Multi-class segmentation of knee MRI based on hybrid attention

Accurate segmentation of knee MRI images is crucial for the diagnosis and treatment of degenerative knee disease and sports injuries. However, many existing methods are hindered by class imbalance and fail to capture the features of small structures, leading to suboptimal segmentation performance.

This study applies hybrid attention and multi-scale feature extraction methods to the problem of multi-class segmentation of knee MRI images and innovates the classic U-Net architecture. Firstly, we propose a Hierarchical Feature Enhancement Fusion (HFEF) module, which is integrated into both the skip connections and the bottleneck layer. This module captures channel and spatial information at multiple levels, enabling the model to efficiently combine local and global features. Secondly, we introduce the Atrous Squeeze Attention (ASA) module, which enables the model to focus on multi-scale features and capture long-range dependencies, thereby improving the segmentation accuracy of complex multi-class structures. Lastly, the loss function is optimized to address the challenges of class imbalance and limited data. The improved loss function enhances the model's ability to learn underrepresented classes, thus enhancing the overall segmentation performance.

We evaluated the proposed method on a knee MRI dataset and compared it with U-Net. HASA-ResUNet achieved a 12.12% improvement in Intersection over Union (IoU) for the low-frequency and small-sized class, the anterior cruciate ligament, and a 3.32% improvement in mean Intersection over Union (mIoU) across all classes.

These results demonstrate that the proposed hybrid attention and multi-scale strategy can effectively address the challenges of class imbalance in knee MRI images, improving the model's overall segmentation performance.