SCKM: Symmetric Co-Skew Moment for User Selection in Federated Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Federated learning (FL) faces challenges with heterogeneous data across clients.
• Existing client-selection heuristics often struggle with data diversity, impacting model convergence.
• Need for advanced metrics to effectively select clients in non-IID FL settings.

Purpose of the Study:

• Introduce the symmetric co-skewness moment (SCKM) as a third-order informational dissimilarity metric.
• Develop a novel client-selection framework for federated learning that minimizes redundancy.
• Enhance convergence speed and stability in FL under high data heterogeneity.

Main Methods:

• Developed SCKM, a metric focusing on complementary client gradients rather than similarity.
• Implemented two operating modes: SCKM-Select for small pools and SCKM-Cluster for large-scale selection using clustering.
• Utilized highly compressed 0.5% gradient summaries for efficient communication.

Main Results:

• SCKM consistently outperformed state-of-the-art client-selection baselines (cosine-similarity, loss-sketch, max-diversity).
• Demonstrated faster and more stable convergence, especially at high levels of data heterogeneity.
• Achieved performance gains without increasing the communication budget.

Conclusions:

• SCKM offers a superior approach to client selection in federated learning compared to similarity-based methods.
• The framework effectively handles data heterogeneity by selecting clients with complementary information.
• SCKM provides a scalable and communication-efficient solution for advanced federated learning deployments.