FedELR: When federated learning meets learning with noisy labels.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) typically assumes high-quality training labels, which is often not the case in real-world applications.
• Noisy labels in FL lead to significant performance degradation, impacting the quality of service in critical areas like medical image classification.

Purpose of the Study:

• To investigate the impact of noisy labels on federated learning by analyzing the early-time training phenomenon (ETP).
• To identify the root cause of performance degradation in noisy FL scenarios.
• To propose a novel framework for mitigating the effects of noisy labels in FL.

Main Methods:

• Analyzing the early training phase dynamics across local and global models in FL.
• Identifying inconsistencies in the early training phases due to varying noisy classes among clients.
• Proposing FedELR, a framework using early learning regularization (ELR) to align local model training phases.

Main Results:

• Demonstrated that early training phases vary among local clients and between local and global models in noisy FL.
• Showed that local clients tend to memorize noisy labels before the global model reaches optimality.
• Validated the effectiveness of FedELR in aligning early training phases and improving model robustness through extensive experiments.

Conclusions:

• The early-time training phenomenon (ETP) is crucial for understanding noisy federated learning.
• Aligning early training phases across local models is a necessary principle for robust noisy FL.
• FedELR offers a simple yet effective solution to enhance federated learning performance in the presence of noisy labels.