A robust sampling technique for realistic distribution simulation in federated learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Medical Informatics

Background:

• Federated Learning (FL) enables training deep learning models on decentralized data, crucial for privacy-sensitive clinical settings.
• Non-Independent and Identically Distributed (non-IID) data, arising from demographic variations across clients, can significantly degrade FL model performance.
• Assessing the impact of non-IID data distributions is vital before implementing large-scale FL in healthcare.

Purpose of the Study:

• To develop and evaluate a novel sampling algorithm for creating realistic, client-biased label distributions.
• To investigate the performance degradation of FL models under simulated non-IID data scenarios.
• To provide an efficient method for analyzing the effects of data heterogeneity in FL.

Main Methods:

• A sampling algorithm was developed to generate data subsets with specified mean and standard deviations from a global distribution.
• Chi-squared and Gini impurity measures were employed for numerical optimization of label distributions across multiple groups.
• The algorithm was applied to a real-world clinical dataset for 3D camera-based weight and height estimation.

Main Results:

• Federated Averaging (FedAvg) training with sampled non-IID data resulted in a performance drop.
• A realistic deterioration of 25.3% for weight and 28.7% for height estimations was observed in the global model.
• The proposed sampling technique demonstrated a significant negative impact compared to a hard data split baseline.

Conclusions:

• Client-biased label distributions in FL settings can substantially impair model training and performance.
• The developed sampling algorithm offers an efficient approach for pre-deployment analysis of non-IID data effects.
• This technique is versatile, applicable to various network architectures, clinical scenarios, and non-IID subpopulations.