Re-Fed+: A Better Replay Strategy for Federated Incremental Learning

Area of Science:

• Machine Learning
• Distributed Systems

Background:

• Federated learning (FL) enables collaborative model training without raw data sharing.
• Traditional FL assumes static data, but real-world scenarios involve incremental data streams.
• Edge clients in FL often have limited storage and computational resources.

Purpose of the Study:

• To address catastrophic forgetting in Federated Incremental Learning (FIL) on resource-constrained edge clients.
• To propose a general, low-cost framework for FIL that caches significant data samples for replay.
• To analyze the effectiveness of sample caching for mitigating forgetting in dynamic data environments.

Main Methods:

• Proposed Re-Fed+ framework for FIL, enabling clients to cache significant previous samples.
• Clients train local models using cached and new task samples.
• Theoretical analysis of sample significance identification and empirical validation.

Main Results:

• Re-Fed+ framework effectively caches important samples for replay.
• The proposed method alleviates catastrophic forgetting in FIL.
• Achieved competitive performance against state-of-the-art methods in empirical evaluations.

Conclusions:

• Re-Fed+ provides an efficient and low-cost solution for FIL on edge devices.
• The framework successfully balances resource constraints with the need to combat catastrophic forgetting.
• Demonstrated the practical viability of sample caching for continuous learning in FL.