Decorrelation of the true and estimated classifier errors in high-dimensional settings

Area of Science:

• Bioinformatics
• Computational Biology
• Statistical Learning

Background:

• Microarray experiments aim to build diagnostic and prognostic models.
• Model validity, crucial in high-dimensional settings, depends on precise error estimation.
• Overfitting (peaking phenomenon) is a challenge due to numerous features and limited samples.

Purpose of the Study:

• To investigate the impact of correlation between true and estimated errors on model validity.
• To understand how high dimensionality affects error estimation precision.
• To analyze the decorrelating effects of high dimensionality in microarray data analysis.

Main Methods:

• Decomposition of the variance of the deviation distribution (estimated error minus true error).
• Analysis of correlation between true and estimated errors under various conditions.
• Comparison of error estimation using synthetic and real data, different feature selection methods, classification rules, and error estimators (leave-one-out cross-validation, k-fold cross-validation, .632 bootstrap).

Main Results:

• High dimensionality significantly decreases the correlation between true and estimated errors.
• The decorrelating effect of high dimensionality is a greater contributor to poor error estimation than increased variance.
• True and estimated errors are more correlated in a known-feature set scenario compared to feature selection or using all features.

Conclusions:

• The precision of error estimation in high-dimensional microarray data is critically affected by reduced correlation between true and estimated errors.
• Feature selection and using all features in high-dimensional settings lead to weaker error-true error correlations.
• Understanding these decorrelating effects is essential for developing reliable diagnostic and prognostic models from microarray data.