FedFask: Fast Sketching Distributed PCA for Large-Scale Federated Data.

Area of Science:

• Machine Learning
• Data Science
• Distributed Computing

Background:

• Federated data presents challenges for Principal Component Analysis (PCA) due to large sample size (n) and dimension (d).
• Existing methods struggle with communication overhead and computational complexity in distributed PCA settings.

Purpose of the Study:

• To develop an efficient and accurate distributed PCA algorithm for ultra-large scale federated data.
• To address the communication and computational bottlenecks in current federated PCA approaches.

Main Methods:

• Introduced FedFask (Fast Sketching for Federated learning), an algorithm with reduced communication ($O(dr)$) and computational complexity ($O(d(np/m+p^{2}+r^{2}))$).
• Employed techniques including fast sketching, orthogonal Procrustes Fixing, and matrix Stiefel manifold averaging.
• Utilized Kolmogorov-Nagumo-type averaging for enhanced eigenspace representation.

Main Results:

• FedFask achieves a learning rate of $O\left(\frac{\kappa _{r}r}{\lambda _{r}}\sqrt{\frac{r^*}{n}}\right)$, matching centralized PCA.
• Demonstrated higher accuracy and lower stochastic variation compared to existing methods.
• Successfully avoided orthogonal ambiguity in eigenspaces and enabled parallel acceleration.

Conclusions:

• FedFask offers a scalable and effective solution for distributed PCA on massive federated datasets.
• The algorithm's efficiency and accuracy make it suitable for real-world large-scale data analysis.
• FedFask provides a robust method for extracting principal components in distributed environments.