Assessing reproducibility and veracity across machine learning techniques in biomedicine: A case study using TCGA...

Area of Science:

• Genomics
• Bioinformatics
• Computational Biology

Background:

• Gene biomarker identification for drug development often lacks clinical validity and reproducibility.
• Statistical learning tools are crucial for genomic data analysis, necessitating an examination of their limitations.

Purpose of the Study:

• To demonstrate methodological gaps in common statistical learning techniques for gene expression analysis.
• To assess the classification ability and reproducibility of machine learning models for gene biomarker detection.

Main Methods:

• Trained six machine learning models on The Cancer Genome Atlas (TCGA) cancer data.
• Evaluated classification ability using standard performance metrics (specificity, sensitivity, precision, F1 score).
• Assessed reproducibility by quantifying the consistency of selected gene classifiers.

Main Results:

• Random forest models exhibited the best overall classification ability.
• Few genes were consistently selected across different methods, indicating poor identifiability and reproducibility.
• Inherent differences in statistical machine learning methods challenge the reproducibility of gene expression discoveries.

Conclusions:

• Statistical machine learning models show significant variation in high-dimensional gene expression data.
• Transparent analysis procedures, including preprocessing, parameterization, and model selection, are essential for clinical validity and utility.