Assessing predictors for new post translational modification sites: A case study on hydroxylation.

Area of Science:

• Biochemistry
• Proteomics
• Bioinformatics

Background:

• Post-translational modifications (PTMs) are crucial for protein function, but many lack sufficient data for reliable predictor development.
• Proline hydroxylation is a significant PTM, particularly in collagen and signaling pathways, making accurate prediction essential.
• Existing PTM predictors often rely on limited training data and may not accurately reflect real-world performance.

Purpose of the Study:

• To evaluate the performance of existing proline hydroxylation site predictors on independent, experimentally determined datasets.
• To compare the accuracy of seven published hydroxylation site predictors.
• To assess the generalizability and reliability of PTM predictors for future experimental design.

Main Methods:

• Benchmarking seven published proline hydroxylation site predictors.
• Utilizing two newly constructed, independent datasets of experimentally determined hydroxylation sites.
• Analyzing predictor performance metrics including specificity, sensitivity, precision, and accuracy on novel data.

Main Results:

• Self-reported performance metrics widely overestimate actual accuracy on independent datasets.
• No tested predictor demonstrated performance significantly better than random chance on new examples.
• High false positive rates and low precision were observed, especially for non-collagenous proteins with less conserved motifs.

Conclusions:

• Current proline hydroxylation site predictors lack sufficient generalizability for detecting new sites.
• Caution is advised when using PTM predictors without independent validation, particularly for signaling-related sites.
• The study highlights the need for more robust predictor development and independent performance evaluations.