Adaptive Tip Selection for DAG-Shard-Based Federated Learning with High Concurrency and Fairness

Area of Science:

• Machine Learning
• Distributed Systems
• Artificial Intelligence

Background:

• Federated learning faces challenges with high-concurrency training of large models and big data.
• Existing alternatives like Directed Acyclic Graph (DAG) and sharding lack detailed consensus designs and shard size impact analysis.
• Blockchain-based federated learning has limitations in training concurrency.

Purpose of the Study:

• To enhance training concurrency and performance in federated learning using a combined DAG and shard approach.
• To investigate the impact of consensus algorithms and shard sizes on federated learning.
• To develop an adaptive algorithm for improved training performance and concurrent control of DAG structure.

Main Methods:

• Combining DAG and sharding techniques for federated learning.
• Designing three tip selection consensus algorithms and an adaptive algorithm.
• Implementing an incentive mechanism to validate model fairness and robustness.
• Adjusting shard and algorithm parameters for concurrent control of DAG scale.

Main Results:

• DSFL demonstrates significant improvements in accuracy (8.19-12.21%) and F1 score (7.27-11.73%) over DAG-FL.
• DSFL shows accuracy gains (7.82-11.86%) and F1 score improvements (8.89-13.27%) compared to Blockchain-FL.
• The proposed model outperforms DAG-FL and Chains-FL on both balanced and imbalanced datasets, proving its effectiveness.

Conclusions:

• DAG-Shard-based Federated Learning (DSFL) offers superior performance in high-concurrency scenarios.
• DSFL provides enhanced fairness and robustness compared to existing federated learning approaches.
• The adaptive algorithm and concurrency control mechanisms effectively manage DAG scale and improve training outcomes.