A Novel Multi-Dimensional Joint Search Method for the Compression of Medical Image Segmentation Models

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Transformers demonstrate strong performance in computer vision tasks.
• Their application in medical image segmentation is growing.
• However, transformers lead to large model parameters, increasing computational resource demands and training time.

Purpose of the Study:

• To explore a flexible and efficient search strategy for identifying optimal subnets within continuous transformer networks.
• To address the challenge of large model parameters and high computational costs associated with transformers in medical image segmentation.

Main Methods:

• A learnable and uniform L1 sparsity constraint is employed to guide the search for efficient subnets.
• The search strategy incorporates factors reflecting global importance across different dimensions of the search space.
• A pixel classification module is integrated to mitigate potential accuracy loss during the subnet search process.

Main Results:

• The proposed method achieved a 30% compression in model parameters and FLOPs (Floating Point Operations Per Second).
• The model demonstrated a slight improvement in accuracy on the Automatic Cardiac Diagnosis Challenge (ACDC) dataset.
• The search process is efficient, requiring only a single round of training.

Conclusions:

• The developed strategy effectively reduces the computational burden of transformer models for medical image segmentation.
• This approach enhances the practicality of using transformers in resource-constrained environments.
• The method offers a balance between model efficiency and diagnostic accuracy in medical imaging applications.