Co-ordinate-based positional embedding that captures resolution to enhance transformer's performance in medical image...

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Vision Transformers (ViTs) excel in computer vision by capturing global dependencies.
• Medical imaging data often requires resampling, risking spatial resolution loss.
• Existing ViT architectures face challenges with variable medical image sizes and resolutions.

Purpose of the Study:

• To propose a coordinate-based embedding for Vision Transformers in medical imaging.
• To overcome the limitations of resampling and resizing in medical image pre-processing.
• To enhance the performance of Transformer-based models in medical image analysis.

Main Methods:

• Developed a coordinate-based embedding encoding image geometry, physical coordinates, and resolution.
• Integrated the proposed embedding into UNETR and SwinUNETR models.
• Evaluated models on an MRI dataset for infarct segmentation using AxTrace and AxADC contrasts.

Main Results:

• Coordinate-based embedding significantly improved mean Dice scores by 6.5% (UNETR) and 7.6% (SwinUNETR).
• The method demonstrated a statistically significant advantage (p-value < 0.0001) over alternative approaches.
• Achieved enhanced performance without compromising spatial resolution or requiring resampling.

Conclusions:

• The proposed coordinate-based pixel-wise positional embedding is a promising solution for Transformer-based medical image analysis.
• This technique effectively utilizes physical coordinate information to boost performance.
• It lays the groundwork for future advancements in positional embedding for medical applications.