Evaluating the predictive accuracy of curated biological pathways in a public knowledgebase

Area of Science:

• Systems Biology
• Bioinformatics
• Molecular Biology

Background:

• Reactome is a manually curated database of human biological pathways.
• Understanding genetic perturbations is crucial for disease research and drug development.
• Evaluating pathway databases for predictive accuracy is essential for their utility.

Purpose of the Study:

• To assess the predictive power of Reactome pathways for functional consequences of genetic perturbations.
• To compare the accuracy of manual curation versus a novel algorithm (MP-BioPath) in predicting perturbation effects.
• To evaluate the utility of Reactome for hypothesis-driven research and experimental standardization.

Main Methods:

• Selected ten cancer-relevant Reactome pathways covering diverse biological processes.
• Defined root input nodes and key pathway outputs for each selected pathway.
• Used pathway-diagram-derived logic graphs to predict perturbation effects via manual inspection (curators) and the MP-BioPath algorithm.

Main Results:

• Analyzed 4968 test cases across 10 pathways, with 847 supported by empirical findings.
• Curators achieved ~81% accuracy, while the MP-BioPath algorithm achieved ~75% accuracy in predicting experimental outcomes.
• Accuracy varied by pathway, with 'RAF/MAP kinase cascade' showing 100% (curator) and 'Mitotic G1 phase and G1/S transition' showing 56% (curator).

Conclusions:

• Reactome pathways demonstrate significant potential for modeling genetic perturbations.
• Both manual curation and the MP-BioPath algorithm show utility, with manual curation yielding higher accuracy in this study.
• This work supports Reactome's role in standardizing pathway activity readouts and generating testable hypotheses.