Can the false-discovery rate be misleading?

Area of Science:

• Proteomics
• Bioinformatics
• Computational Biology

Background:

• The decoy-database approach is the standard for evaluating confidence in shotgun proteomic identification.
• This method uses decoy (random) sequences to estimate the false-discovery rate (FDR).
• Overfitting can occur, leading to inflated identification numbers that do not reflect the true FDR.

Purpose of the Study:

• To demonstrate that the decoy-database approach can be susceptible to overfitting.
• To introduce a novel method, the semi-labeled decoy approach, to detect and address overfitting.
• To improve the reliability of protein identifications in shotgun proteomics.

Main Methods:

• Implementation of the standard decoy-database approach in shotgun proteomic data analysis.
• Development and application of a modified semi-labeled decoy strategy.
• Statistical analysis to determine the presence and extent of overfitting.

Main Results:

• The study reveals that apparent good results from the decoy-database approach can be artifacts of overfitting.
• Overfitting inflates protein identification numbers, misrepresenting the actual false-discovery rate.
• The semi-labeled decoy approach successfully identifies overfitted results.

Conclusions:

• The conventional decoy-database approach may yield misleadingly high confidence scores due to overfitting.
• The semi-labeled decoy approach provides a statistically sound method to detect and mitigate overfitting.
• This advancement enhances the accuracy and reliability of protein identifications in shotgun proteomics.