UNesT: Local spatial representation learning with hierarchical transformer for efficient medical segmentation.

Area of Science:

• Medical Image Analysis
• Computer Vision
• Artificial Intelligence

Background:

• Transformer models excel in computer vision and medical imaging but struggle with preserving positional information in 3D medical image segmentation.
• Existing methods lack robustness and efficiency for complex tasks like segmenting numerous tissue classes or interconnected structures.

Purpose of the Study:

• To introduce UNesT, a novel 3D medical image segmentation method utilizing a hierarchical transformer encoder.
• To address limitations in positional information preservation and improve efficiency and robustness in heavy-duty medical segmentation tasks.

Main Methods:

• Developed UNesT, a transformer-based model with a simplified, faster-converging encoder for hierarchical aggregation of patch sequences.
• Implemented local communication among spatially adjacent patches to preserve positional information.

Main Results:

• UNesT achieved state-of-the-art performance across multiple challenging datasets with diverse modalities, anatomies, and tissue classes.
• Successfully performed whole brain segmentation with 133 tissue classes in a single network, outperforming ensemble methods.
• Improved mean DSC scores on Colin and CANDI datasets, demonstrating enhanced segmentation accuracy.

Conclusions:

• UNesT offers a robust and efficient solution for 3D medical image segmentation, particularly for complex and large-scale tasks.
• The hierarchical transformer approach effectively preserves crucial positional information, leading to superior performance and generalizability.