Integration of large language models and federated learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Data Science

Background:

• The expanding parameter size of large language models (LLMs) necessitates high-quality data, creating a data scarcity challenge.
• Federated learning (FL) offers a privacy-preserving approach to training models on decentralized data.
• The inherent strengths of LLMs in task generalization and FL in data privacy create a synergistic research opportunity.

Purpose of the Study:

• To comprehensively review the integration of LLMs and FL.
• To propose a framework for understanding the fusion of LLM and FL sub-technologies.
• To explore practical applications and future research directions in this interdisciplinary domain.

Main Methods:

• A systematic review of existing research on LLM and FL integration.
• Development of a three-part research framework: LLM sub-technologies with FL, FL sub-technologies with LLMs, and overall merger.
• Analysis of typical applications, advantages, challenges, and future directions.

Main Results:

• Identified a growing research interest in the complementarity of LLMs and FL.
• Detailed a research framework categorizing integration approaches.
• Summarized current applications, benefits, and challenges of combined LLM-FL systems.

Conclusions:

• The integration of LLMs and FL presents a promising avenue for overcoming data limitations in AI.
• Further research is needed to address challenges and unlock the full potential of this synergy.
• Practical applications in healthcare, finance, and education highlight the transformative impact of combined LLM-FL approaches.