Federated generative prompt learning with vision foundation models: universal efficient multi-center medical image...

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Collaborative AI

Background:

• Federated AI enables multi-center medical collaboration but faces challenges like communication costs and data heterogeneity.
• Foundation models (FMs) show promise for medical AI due to their adaptability and generalization.
• Existing methods struggle with efficiency and data limitations in federated medical image analysis.

Purpose of the Study:

• To introduce Federated Generative Prompt Learning (Fed-GPL), a universal and efficient framework for multi-center medical image analysis.
• To address limitations in federated medical AI, including communication costs, data scarcity, and heterogeneity.
• To enable precise patient-specific medical diagnosis through customized prompts.

Main Methods:

• Developed Fed-GPL, a framework that collaboratively trains a prompt generator for patient-specific prompt creation.
• Integrated Fed-GPL with various vision foundation models, including Vision Transformer (ViT) and Segment Anything (SAM).
• Applied Fed-GPL to diverse medical imaging tasks: diabetic retinopathy, melanoma classification, polyp segmentation, and prostate segmentation.

Main Results:

• Fed-GPL significantly outperforms traditional models and full fine-tuning approaches.
• Achieved high performance with minimal parameter training (8.26% for classification, 6.55% for segmentation).
• Demonstrated rapid convergence within 15 communication rounds and maintained performance with only 5% of training data in low-resource settings.

Conclusions:

• Fed-GPL offers a universal, efficient, and adaptable solution for federated medical image analysis.
• The framework effectively overcomes challenges in multi-center collaboration, data scarcity, and heterogeneity.
• Fed-GPL enables precise diagnosis and robust performance across various medical tasks and resource constraints.