Consistency and overfitting of multi-omics methods on experimental data.

Area of Science:

• Computational Biology
• Bioinformatics
• Genomics

Background:

• Understanding interactions across biological modalities (e.g., RNA, chromatin) is key to deciphering biological processes.
• The proliferation of multi-omics datasets necessitates robust methods for identifying shared variation.
• Evaluating unsupervised multi-omics integration methods for consistency has been challenging.

Purpose of the Study:

• To compare the performance of sparse multiple canonical correlation analysis (Sparse mCCA), angle-based joint and individual variation explained (AJIVE), and multi-omics factor analysis (MOFA).
• To assess method consistency and overfitting using a cross-validation framework across different sample sizes.
• To provide a reproducible framework for evaluating multi-omics integration techniques.

Main Methods:

• A cross-validation approach was employed to evaluate Sparse mCCA, AJIVE, and MOFA.
• Performance was assessed using both large and small sample-sized datasets.
• A permuted null dataset was utilized to detect overfitting within the evaluation framework.

Main Results:

• In large-sample settings, all tested methods (Sparse mCCA, AJIVE, MOFA) exhibited consistency and no significant overfitting.
• In small-sample settings, AJIVE demonstrated superior stability compared to Sparse mCCA and MOFA.
• The developed framework successfully identified overfitting tendencies in small-sample scenarios.

Conclusions:

• AJIVE emerges as a more reliable method for multi-omics data integration when dealing with limited sample sizes.
• The proposed evaluation framework and R package facilitate rigorous assessment of multi-omics analysis methods.
• Consistent and stable results are critical for advancing our understanding of multi-modal biological data.