Fast R Functions for Robust Correlations and Hierarchical Clustering.

Area of Science:

• Bioinformatics
• Computational Biology
• Data Science

Background:

• High-throughput biological data analysis often involves large correlation matrix calculations and clustering of numerous objects.
• Standard R functions for Pearson correlation are inefficient with missing data, and hierarchical clustering (hclust) has cubic time complexity.
• Existing computational bottlenecks hinder the analysis of large biological datasets.

Purpose of the Study:

• To develop and implement computationally efficient algorithms for correlation calculation and hierarchical clustering in R.
• To accelerate the analysis of large biological datasets, particularly those with missing values.
• To provide robust and fast alternatives to standard R functions for correlation and clustering.

Main Methods:

• Implementation of a parallelized Pearson correlation calculation optimized for datasets with missing values.
• Development of a fast biweight midcorrelation function (bicor) with similar performance gains.
• Introduction of an optimized hierarchical clustering algorithm (flashClust) with quadratic time complexity.

Main Results:

• Substantial speedups achieved for Pearson correlation calculations on data with missing entries.
• Comparable speed improvements for biweight midcorrelation calculations.
• Significant time savings in hierarchical clustering for large datasets using flashClust compared to hclust.

Conclusions:

• The new R functions (cor and bicor in WGCNA, flashClust) offer significant computational advantages for large-scale biological data analysis.
• These optimized methods facilitate more efficient handling of missing data and large object clustering.
• The freely available WGCNA and flashClust packages enhance the capabilities of R for bioinformatics research.