Parallel GPU implementation of iterative PCA algorithms

Area of Science:

• Multivariate statistical analysis
• Computational statistics
• High-performance computing

Background:

• Principal Component Analysis (PCA) is crucial for multivariate data analysis.
• Standard NIPALS-PCA algorithm struggles with orthogonality for large datasets, limiting component estimation.
• Existing methods face challenges in efficiently handling large-scale PCA computations.

Purpose of the Study:

• Introduce a novel Gram-Schmidt orthogonalization PCA (GS-PCA) algorithm.
• Address the orthogonality limitations of the NIPALS-PCA method.
• Investigate and compare GPU parallel implementations for enhanced computational efficiency.

Main Methods:

• Developed a Gram-Schmidt orthogonalization-based PCA algorithm (GS-PCA).
• Implemented parallel versions of both NIPALS-PCA and GS-PCA using Graphics Processing Units (GPUs).
• Utilized CUBLAS for NVIDIA GPU optimization and CBLAS for CPU optimization.

Main Results:

• GS-PCA overcomes the orthogonality loss issue inherent in NIPALS-PCA.
• GPU parallel implementations demonstrate significant speed improvements over CPU versions.
• Optimized GPU versions achieved up to 12 times faster computation compared to CPU-based methods.

Conclusions:

• GS-PCA offers a robust alternative to NIPALS-PCA, preserving orthogonality.
• GPU parallelization is highly effective for accelerating PCA computations.
• The proposed GPU-accelerated GS-PCA is suitable for efficient large-scale multivariate data analysis.