High performance data integration for large-scale analyses of incomplete Omic profiles using Batch-Effect Reduction...

Area of Science:

• Bioinformatics
• Computational Biology
• Data Science

Background:

• High-throughput omic data frequently contains missing values and batch-effects, impeding quantitative analysis and dataset integration.
• Existing methods struggle with incomplete omic profiles, limiting the scope of integrated analyses.

Purpose of the Study:

• Introduce Batch-effect Reduction Trees (BERT), a novel high-performance method for integrating incomplete omic data.
• Address limitations of current data integration techniques for omics and other datatypes.

Main Methods:

• Developed BERT, a method designed for efficient data integration of omic profiles with missing values.
• Characterized BERT on large-scale integration tasks involving up to 5000 datasets across various omic types (proteomics, transcriptomics, metabolomics) and datatypes (clinical data).
• Leveraged multi-core and distributed-memory systems for computational efficiency.

Main Results:

• BERT retains significantly more numeric values compared to existing methods.
• Achieved up to an 11-fold runtime improvement by utilizing parallel processing capabilities.
• Improved data integration quality, evidenced by up to a 2-fold increase in average silhouette width, by considering covariates and reference measurements.

Conclusions:

• BERT offers a robust and scalable solution for integrating incomplete omic datasets, enhancing quantitative comparability.
• The method demonstrates broad applicability across diverse omic types and other data modalities.
• BERT significantly outperforms existing methods in data retention, runtime, and integration quality.