Federated Gradient Averaging for Multi-Site Training with Momentum-Based Optimizers.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Medical Informatics

Background:

• Multi-site training for artificial neural networks is crucial in medical machine learning due to data sharing challenges.
• Existing methods like weight averaging and cyclic weight transfer involve theoretical compromises.

Purpose of the Study:

• To implement and evaluate Federated Gradient Averaging (FGA), a novel federated learning variant.
• To demonstrate FGA's mathematical equivalence to single-site centralized training.
• To compare FGA against existing multi-site training techniques.

Main Methods:

• Implemented Federated Gradient Averaging (FGA), a federated learning approach.
• Evaluated FGA on a simulated multi-site MNIST dataset for digit classification.
• Assessed FGA on a real multi-site head CT dataset for hemorrhage segmentation.
• Compared FGA against single-site training, Federated Weight Averaging (FWA), and cyclic weight transfer.

Main Results:

• FGA training on MNIST yielded a weight set identical to centralized single-site training.
• In head CT hemorrhage segmentation, FGA outperformed both FWA and cyclic weight transfer.
• FGA's superior performance is attributed to its effective use of momentum-based optimization.

Conclusions:

• Federated Gradient Averaging (FGA) offers a theoretically sound and practically effective solution for multi-site artificial neural network training.
• FGA overcomes limitations of current methods by leveraging momentum optimization, particularly in complex medical imaging tasks.
• This approach facilitates collaborative model development without compromising data privacy or performance.