Deep-Learning-Derived Evaluation Metrics Enable Effective Benchmarking of Computational Tools for Phosphopeptide...

Area of Science:

• Biochemistry
• Computational Biology
• Proteomics

Background:

• Tandem mass spectrometry (MS/MS) enables global phosphorylation analysis.
• Existing computational pipelines for phosphoproteomics show significant discrepancies in phosphopeptide identification and site localization.
• A lack of robust evaluation metrics hinders the comparison and improvement of these pipelines, especially for real-world cancer study data.

Purpose of the Study:

• To address the critical need for benchmarking computational pipelines in MS/MS-based phosphoproteomics.
• To investigate and validate deep-learning-derived features as reliable evaluation metrics for phosphopeptide identification and phosphosite localization.
• To provide a standardized approach for comparing the performance of different computational pipelines using real-world phosphoproteomic datasets.

Main Methods:

• Investigated three deep-learning features: phosphosite probability (MusiteDeep), Delta Retention Time (RT) (AutoRT), and spectral similarity (pDeep2).
• Evaluated these features using a synthetic peptide dataset to assess their ability to distinguish correct from incorrect peptide-spectrum matches (PSMs), including those with incorrect localization.
• Applied the validated features to benchmark diverse computational pipelines on multiple phosphoproteomic datasets.

Main Results:

• Delta RT and spectral similarity effectively discriminated between correct and incorrect PSMs, even when only phosphosite localization was incorrect.
• The three deep-learning-derived features proved useful for benchmarking the performance of various computational pipelines across different phosphoproteomic datasets.
• Demonstrated the utility of these metrics in comparing pipeline accuracy for both peptide identification and site localization.

Conclusions:

• Deep-learning-derived features (phosphosite probability, Delta RT, spectral similarity) serve as effective metrics for benchmarking phosphoproteomic computational pipelines.
• These metrics enable users to select appropriate pipelines and parameters for phosphoproteomics data analysis.
• The study provides guidance for developers to enhance computational methods for improved phosphoproteomics research.