AsynDBT: asynchronous distributed bilevel tuning for efficient in-context learning with large language models.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Natural Language Processing

Background:

• Cloud-based large language models (LLMs) require costly prompt tuning due to parameter and gradient agnosticism.
• In-context learning (ICL) adapts LLMs without parameter updates but is limited by sensitive, hard-to-share data.
• Federated learning (FL) enables privacy-preserving collaborative training but faces challenges with stragglers and heterogeneous data in ICL.

Purpose of the Study:

• To develop a novel algorithm addressing the limitations of existing federated learning approaches for in-context learning in large language models.
• To enhance downstream task performance by optimizing both in-context learning samples and prompt fragments.
• To provide a privacy-preserving and adaptable solution for distributed LLM training in heterogeneous environments.

Main Methods:

• Proposed an asynchronous distributed bilevel tuning (AsynDBT) algorithm.
• Optimized in-context learning samples and prompt fragments based on LLM feedback.
• Implemented a distributed architecture for privacy and adaptability.
• Provided theoretical convergence guarantees for the algorithm.

Main Results:

• AsynDBT enhances downstream task performance by optimizing ICL samples and prompts.
• The distributed architecture ensures privacy protection and adaptability to heterogeneous computing environments.
• Extensive experiments on benchmark datasets validate the effectiveness and efficiency of AsynDBT.

Conclusions:

• AsynDBT offers an effective and efficient solution for privacy-preserving federated learning with in-context learning in large language models.
• The algorithm successfully addresses straggler and data heterogeneity issues in federated in-context learning.
• AsynDBT demonstrates strong performance and adaptability across diverse datasets.