Visual prompt tuning for task-flexible medical image synthesis.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Medical image synthesis is crucial for applications like modality translation, denoising, and super-resolution.
• Traditional methods require separate models for each task, leading to inefficiency and limited scope.
• Existing approaches struggle to accommodate the diverse range of tasks in medical image synthesis.

Purpose of the Study:

• To develop a single, versatile model for various medical image synthesis tasks.
• To improve the efficiency and applicability of medical image synthesis.
• To enable task-agnostic synthesis through prompt tuning.

Main Methods:

• Introduced a task-agnostic medical image synthesis model.
• Utilized prompt tuning with a diffusion model to efficiently fine-tune large pretrained models.
• Developed a single model capable of handling multiple synthesis tasks and input-output combinations.

Main Results:

• The model successfully performed denoising, translation, super-resolution, and tumor inpainting on brain MRI and abdominal CT.
• Achieved state-of-the-art performance across all evaluated tasks, indicated by FID scores.
• Demonstrated high quantitative metrics: PSNR (up to 30.30) and SSIM (up to 0.932) for various tasks, and FID (16.18) for inpainting.

Conclusions:

• Proposed a task-agnostic medical image synthesis method using prompt tuning.
• The method allows specifying synthesis task, modality, and organ via prompts.
• The approach is extensible to other modalities and organs, with code available.