Dual prompt personalized federated learning in foundation models

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Personalized federated learning (PFL) addresses data heterogeneity and privacy but struggles with limited local data.
• Foundation models like CLIP offer strong feature extraction for limited data scenarios but are underutilized in federated settings.
• Integrating new clients and enabling prediction on novel data sources remain unresolved challenges in PFL.

Purpose of the Study:

• To propose a novel framework, Dual Prompt Personalized Federated Learning (DP2FL), to overcome limitations of PFL with insufficient local data.
• To enhance model generalization and adaptability to diverse data distributions in federated environments.
• To enable prediction on new data sources and seamless integration of new clients without full retraining.

Main Methods:

• Introduced a Dual Prompt Personalized Federated Learning (DP2FL) framework incorporating dual prompts and an adaptive aggregation strategy.
• Combined global task awareness with local data-driven insights for model training.
• Developed a global model for prediction on new data sources and client integration.

Main Results:

• DP2FL demonstrated effective generalization and adaptability in highly heterogeneous environments.
• The prompt design and aggregation strategy proved effective, validating the framework's approach.
• The framework successfully enabled prediction on novel data sources and seamless integration of new clients.

Conclusions:

• DP2FL effectively addresses the challenge of limited local data in personalized federated learning by leveraging foundation models.
• The proposed dual prompt and adaptive aggregation strategies enhance model performance and adaptability.
• DP2FL offers a scalable solution for integrating new clients and utilizing novel data sources in federated learning systems.