Fast and accurate out-of-core PCA framework for large scale biobank data.

Area of Science:

• Genomics
• Machine Learning
• Bioinformatics

Background:

• Principal Component Analysis (PCA) is crucial for dimensionality reduction in large datasets.
• Existing PCA methods face challenges with increasing data size, demanding faster and memory-efficient solutions.

Purpose of the Study:

• To introduce PCAone, a novel PCA algorithm utilizing randomized singular value decomposition (RSVD).
• To enhance PCA performance through window-based optimization, out-of-core, and multithreaded implementations.

Main Methods:

• Development of a novel RSVD algorithm within the PCAone framework.
• Implementation of window-based optimization for accelerated convergence and accuracy.
• Integration of out-of-core and multithreaded capabilities for existing Implicitly Restarted Arnoldi Method (IRAM) and RSVD.

Main Results:

• PCAone demonstrates significantly faster computation times compared to existing methods.
• The method maintains accuracy comparable to the slower IRAM approach.
• Analysis of UK Biobank data (0.5M individuals) for top 40 PCs completed in 9 hours with <20 GB memory.
• Single-cell RNA sequencing data (1.3M cells) analysis for top 40 PCs completed in 49 minutes, a 10x improvement.

Conclusions:

• PCAone provides a fast and memory-efficient solution for large-scale PCA.
• The tool accurately captures essential biological structures in genomics and single-cell data.
• PCAone represents a significant advancement over state-of-the-art PCA tools for big data analysis.