Pro-Tuning: Prototype Tuning of Foundation Models for Volumetric Medical Image Segmentation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Accurate volumetric medical image segmentation is vital for diagnosis, treatment planning, and surgical guidance.
• Foundation models show promise in medical image segmentation but struggle with direct application or prompt-based fine-tuning for specific tasks.
• Existing fine-tuning methods often require additional prompts, limiting their efficiency and applicability.

Purpose of the Study:

• To propose a novel, prompt-free method called Pro-Tuning for enhancing medical foundation models in volumetric segmentation.
• To improve the performance of foundation models on specific medical segmentation tasks, especially in complex regions with multiple organs.
• To validate the effectiveness of Pro-Tuning across diverse medical imaging datasets and anatomical regions.

Main Methods:

• Developed Pro-Tuning, a method utilizing a pre-trained Prototype Insight Network to extract semantic prototype features.
• Introduced a Prototype Projection Network to adapt prototype features using target position-encoded image embeddings for specific tasks.
• Validated the approach on 13 medical datasets covering brain, neck, chest, and abdomen regions.

Main Results:

• Pro-Tuning significantly improved the performance of medical foundation models in volumetric segmentation tasks without requiring additional prompts.
• The method demonstrated superior performance compared to other foundation model fine-tuning techniques on ten major organs.
• Achieved an average Dice score of 83.26% across the evaluated datasets.

Conclusions:

• Pro-Tuning offers a simple yet efficient solution for tuning medical foundation models for volumetric segmentation.
• The prompt-free nature and adaptive feature tailoring make Pro-Tuning highly effective for diverse medical imaging applications.
• This method advances the utility of foundation models in clinical settings, enhancing diagnostic and treatment planning capabilities.