Related Experiment Videos
Appropriateness of some resampling-based inference procedures for assessing performance of prognostic classifiers
Lara Lusa1, Lisa M McShane, Michael D Radmacher
1Department of Experimental Oncology, Istituto Nazionale per lo Studio e la Cura dei Tumori, Milano, Italy. lara.lusa@ifom-ieo-campus.it
Statistics in Medicine
|June 7, 2006
Summary
Misapplying statistical tests to gene-expression classifiers can inflate type I error rates, leading to inaccurate assessments of predictive accuracy. Careful interpretation is crucial for reported prognostic classifier performance in clinical studies.
Area of Science:
- Bioinformatics
- Biostatistics
- Clinical Genomics
Background:
- Gene-expression microarray profiling is widely used in clinical studies to predict patient disease outcomes.
- Preliminary validation of predictive classifiers often uses the same dataset for derivation and testing.
Purpose of the Study:
- To evaluate the impact of applying standard statistical inference procedures to assess the significance of gene-expression classifier performance.
- To identify potential biases and inflated error rates in the validation of predictive classifiers.
Main Methods:
- Demonstration of inflated Type I error rates when standard statistical inference is naively applied to cross-validated predicted outcomes.
- Analysis of confidence interval coverage probabilities under null and alternative hypotheses for classifier performance.
Main Results:
- Naïve application of standard statistical inference procedures leads to greatly inflated Type I error rates under null situations.
- Confidence interval coverage probabilities are often too low for small to moderate associations, potentially overestimating classifier performance.
- For very large associations, coverage probabilities approach intended values, but caution is still advised.
Conclusions:
- Standard statistical inference procedures should be applied with caution when assessing gene-expression classifier performance.
- Overstated claims of exceptional prognostic classifier performance in biomedical literature may result from inappropriate statistical methods.
- Researchers should exercise caution in interpreting and reporting the predictive accuracy of gene-expression classifiers.