FedMed: A Federated Learning Framework for Language Modeling.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Natural Language Processing

Background:

• Federated learning (FL) enables privacy-preserving training on decentralized data for mobile devices.
• Mobile keyboard prediction, a language modeling task, requires efficient AI with high data privacy.
• Existing FL methods struggle with server-side aggregation and high communication costs.

Purpose of the Study:

• To propose a novel Federated Mediation (FedMed) framework to tackle aggregation and communication challenges in federated optimization for mobile keyboard prediction.
• To enhance the efficiency and privacy of AI applications on mobile devices through improved federated learning techniques.

Main Methods:

• Developed the Federated Mediation (FedMed) framework incorporating adaptive aggregation, a mediation incentive scheme, and a topK strategy.
• Evaluated FedMed's performance using perplexity and communication rounds on three diverse datasets: Penn Treebank, WikiText-2, and Yelp.

Main Results:

• The FedMed framework demonstrated robust performance in federated optimization for mobile keyboard prediction.
• FedMed significantly outperformed baseline approaches in terms of perplexity and communication efficiency.

Conclusions:

• The proposed FedMed framework effectively addresses key challenges in federated learning for mobile keyboard prediction, namely model aggregation and communication costs.
• FedMed offers a promising solution for privacy-preserving AI applications, balancing performance with reduced communication overhead.