Replica tree-based federated learning using limited data

Area of Science:

• Machine Learning
• Federated Learning
• Artificial Intelligence

Background:

• Deep neural networks require large datasets for optimal performance.
• Centralized training strategies are well-established, but federated learning with limited data and clients is underexplored.
• Real-world applications, particularly in healthcare, often involve constrained numbers of participating clients and limited data.

Purpose of the Study:

• To propose a novel federated learning framework, RepTreeFL, designed for scenarios with limited data and a small number of clients.
• To address the challenge of learning effectively when both data volume and client participation are constrained.
• To enable robust model training in resource-limited federated environments.

Main Methods:

• Introduced RepTreeFL, a federated learning framework utilizing client model replicas.
• Replicated clients by copying model architecture and perturbing local data distributions to create model diversity.
• Implemented a diversity-based tree aggregation strategy, dynamically updating weights based on model discrepancy.
• Leveraged hierarchical client network structures (original and virtual) and model diversity for aggregation.

Main Results:

• Demonstrated the effectiveness of RepTreeFL in learning from limited data and a small number of clients.
• Showcased superior performance of RepTreeFL compared to existing methods in constrained settings.
• Validated the framework across diverse tasks (graph generation, image classification) and data types (binary, multi-class).
• Confirmed effectiveness with both homogeneous and heterogeneous model architectures.

Conclusions:

• RepTreeFL successfully enables effective federated learning with limited data and clients.
• The replica concept and diversity-based tree aggregation are key to overcoming data and client constraints.
• The proposed framework offers a promising solution for practical federated learning applications in resource-limited environments.