Measuring consistency among gene set analysis methods: A systematic study.

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• Gene set analysis (GSA) is crucial for interpreting gene expression data.
• Numerous GSA methods exist, raising concerns about method-dependent results.

Purpose of the Study:

• To systematically evaluate the impact of different GSA methods on results.
• To assess the reproducibility and biological relevance of GSA findings across various datasets and methods.

Main Methods:

• Analysis of 13 popular GSA methods using 6 diverse gene expression datasets (DNA microarray and RNA-Seq).
• Comparison of the number of reported gene sets, bias towards larger sets, and agreement among top significant gene sets (20 and 100).
• Biological validation using a juvenile idiopathic arthritis (JIA) dataset.

Main Results:

• Significant variation in the number of gene sets identified (up to 2 orders of magnitude) and bias towards large gene sets by some methods.
• Substantial disagreement in top significant gene sets across methods and even across datasets of the same condition.
• GAGE, PAGE, and ORA demonstrated higher reproducibility for top gene sets.
• GAGE, GSEA, ORA, and PAGE showed varying degrees of biological relevance for JIA.

Conclusions:

• The choice of gene set analysis method profoundly influences biological insights derived from gene expression data.
• Method selection is critical for reliable and reproducible GSA results.
• GAGE, PAGE, and ORA are recommended for higher reproducibility, with GAGE showing promise in biological relevance.