Learning From Each Other: Generalized Federated Incremental Semantic Segmentation

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Federated learning (FL) enhances semantic segmentation by reducing annotation costs through decentralized training.
• Existing FL methods for semantic segmentation suffer from catastrophic forgetting when clients encounter new classes in streaming data.
• Heterogeneous forgetting across clients is exacerbated by the irregular participation of new clients with novel classes.

Purpose of the Study:

• To propose a novel Hierarchical Forgetting Alleviation (HFA) model to address catastrophic forgetting in FL-based semantic segmentation.
• To ensure continuous learning of new categories while retaining knowledge of old categories across all local clients.
• To mitigate both within-client and across-client forgetting in dynamic FL environments.

Main Methods:

• Developed a confidence-regularized pseudo labeling strategy for class-balanced soft pseudo labels to alleviate within-client forgetting.
• Designed a graph-induced relation matching loss and a forgetting-balanced gradient propagation module to handle old class relations.
• Implemented a task detection module and adaptive DBSCAN clustering to address inter-client heterogeneous forgetting and manage global model updates.

Main Results:

• The proposed HFA model effectively alleviates class-imbalanced forgetting within local clients.
• The model successfully tackles ambiguous inter-class relations and class-imbalanced gradient propagation.
• Experiments demonstrate the superiority of HFA over existing methods on multiple datasets.

Conclusions:

• The HFA model provides a robust solution for catastrophic forgetting in federated semantic segmentation.
• The proposed methods enable effective knowledge retention and transfer in dynamic, heterogeneous FL settings.
• HFA ensures that all clients can learn from each other while continuously adapting to new data and classes.