Classification based upon gene expression data: bias and precision of error rates.

Area of Science:

• Bioinformatics
• Computational Biology
• Statistical Learning

Background:

• Gene expression data offers numerous predictors for classifying tissue samples into disease states.
• Estimating classifier predictive error rates commonly uses cross-validation techniques.
• Potential biases in reporting error rate estimates, including optimization and selection biases, require careful consideration.

Purpose of the Study:

• To investigate and address issues of interpretation and potential bias in reporting error rate estimates from gene expression data classification.
• To introduce a novel method for detecting bias using the permutation mean.
• To recommend best practices for accurate error rate estimation in gene expression studies.

Main Methods:

• Investigated optimization and selection biases, sampling effects, and measures of misclassification rate.
• Employed two-level external cross-validation and label permutations for bias detection and avoidance.
• Compared single-level cross-validation with a test set against two-level cross-validation for accuracy.

Main Results:

• Reporting optimal error rates can introduce downward optimization bias (3-5% observed in existing studies).
• Label permutations effectively detect bias, and two-level external cross-validation avoids it.
• Two-level cross-validation provides more accurate error rate estimates than single-level methods.

Conclusions:

• Bias in error rate estimation is prevalent in gene expression classification studies.
• Two-level external cross-validation and permutation-based bias detection are crucial for reliable results.
• Recommends reporting class error rates, conditional risk, and baseline error rates for comprehensive evaluation.