Toward a measure of classification complexity in gene expression signatures

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• Gene expression signatures are crucial for predicting disease outcomes, with successes in leukemia and breast cancer.
• Challenges in gene signature reliability arise from small sample sizes and high data dimensionality.
• Existing methods often fail to generalize across different datasets, limiting their clinical utility.

Purpose of the Study:

• To explore classification complexity as a limit to predictive accuracy in gene expression datasets.
• To identify reliable measures for estimating the maximum attainable accuracy of gene signatures.
• To reduce the need for extensive classifier evaluation by predicting performance limits.

Main Methods:

• Evaluated three measures of classification complexity.
• Utilized two independent gene expression datasets (lung and colorectal cancer) with three outcomes each.
• Tested four classifiers with t-test feature selection, analyzing Fisher's discriminant ratio.

Main Results:

• Fisher's discriminant ratio strongly correlated with classification complexity and accuracy (R(2)=0.78).
• Predicting gender was identified as a low-complexity problem with high accuracy.
• Clinically relevant endpoints presented higher complexity and lower classification accuracies.

Conclusions:

• Classification complexity is a key factor influencing gene signature predictive accuracy.
• Fisher's discriminant ratio serves as a reliable metric for assessing classification problem complexity.
• Estimating maximum attainable accuracy via complexity measures can optimize the development of robust gene expression signatures.