Benchmarking gene ontology function predictions using negative annotations.

Area of Science:

• Genomics and Bioinformatics
• Computational Biology
• Protein Function Prediction

Background:

• Automated gene function characterization is crucial due to the increasing volume and diversity of sequenced species.
• Current methods, including the Critical Assessment of protein Function Annotation (CAFA), rely heavily on automated electronic methods.
• Existing benchmarks may underestimate precision due to an incomplete handling of the open world assumption (OWA) and a lack of negative experimental annotations.

Purpose of the Study:

• To introduce a novel benchmark for evaluating protein function annotation algorithms that is compliant with the open world assumption (OWA).
• To address the systematic underestimation of precision in existing benchmarks by incorporating negative experimental annotations.

Main Methods:

• Developed a new benchmark utilizing a balanced test set of both positive and negative protein annotations.
• Derived negative annotations from expert-curated protein family annotations on phylogenetic trees.
• Tested the benchmark using baseline methods (naïve, BLAST) and orthology-based methods.

Main Results:

• The new benchmark significantly increases the average information content of negative annotations.
• The OWA-compliant benchmark provides a more accurate assessment of prediction precision.
• The benchmark was successfully tested with various established annotation methods.

Conclusions:

• The proposed benchmark offers a more robust evaluation of protein function annotation tools.
• This new benchmark can serve as a valuable complement to existing benchmarks in future CAFA experiments.
• Improved benchmarking is essential for accurate gene function characterization in the era of large-scale sequencing.