Related Experiment Video
Updated: Jun 22, 2026
07:35
A Knowledge Graph Approach to Elucidate the Role of Organellar Pathways in Disease via Biomedical Reports
Published on: October 13, 2023
The impact of incomplete knowledge on evaluation: an experimental benchmark for protein function prediction
Curtis Huttenhower1, Matthew A Hibbs, Chad L Myers
1Department of Computer Science, Princeton University, Princeton, NJ 08540-5233, USA.
Bioinformatics (Oxford, England)
|June 30, 2009
Summary
Creating a reliable protein function gold standard is crucial for accurate machine learning. This study developed a yeast mitochondria benchmark, revealing that incomplete annotations hinder reliable performance evaluation of computational methods.
Area of Science:
- Genomics
- Computational Biology
- Biochemistry
Background:
- Genomic data expansion necessitates robust protein function prediction and biological network inference.
- Supervised machine learning requires a reliable gold standard for protein function assessment.
- Incomplete gene annotations in model organisms challenge the biological reliability of performance evaluations.
Purpose of the Study:
- To construct and analyze an experimentally validated gold standard for protein function prediction.
- To assess the impact of incomplete functional annotations on machine learning performance evaluation.
- To provide a benchmark for yeast mitochondria to improve future comparative analyses.
Main Methods:
- Comprehensive validation of protein function predictions for mitochondrion biogenesis in Saccharomyces cerevisiae.
- Construction and analysis of an experimentally based gold standard.
- Evaluation of machine learning generalization from incomplete data.
Main Results:
- Machine learning methods can generalize and predict novel biology even with incomplete gold standards.
- Incomplete functional annotations negatively impact the evaluation of machine learning performance.
- Computational approaches performed better than anticipated with sparse data, but relative comparisons are problematic.
Conclusions:
- Incomplete knowledge leads to underestimated method performance and misleading evaluations.
- A benchmark gold standard for yeast mitochondria is provided to mitigate these effects.
- Accurate evaluation of protein function prediction requires a high-quality, experimentally validated gold standard.
Related Concept Videos
02:04Protein-protein Interfaces
Many proteins form complexes to carry out their functions, making protein-protein interactions (PPIs) essential for an organism's survival. Most PPIs are stabilized by numerous weak noncovalent chemical forces. The physical shape of the interfaces determines the way two proteins interact. Many globular proteins have closely-matching shapes on their surfaces, which form a large number of weak bonds. Additionally, many PPIs occur between two helices or between a surface cleft and a polypeptide...
02:26Protein Networks
An organism can have thousands of different proteins, and these proteins must cooperate to ensure the health of an organism. Proteins bind to other proteins and form complexes to carry out their functions. Many proteins interact with multiple other proteins creating a complex network of protein interactions.
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...
01:49Conserved Binding Sites
Many proteins’ biological role depends on their interactions with their ligands, small molecules that bind to specific locations on the protein known as ligand-binding sites. Ligand-binding sites are often conserved among homologous proteins as these sites are critical for protein function.
Binding sites are often located in large pockets, and if their location on a protein’s surface is unknown, it can be predicted using various approaches. The energetic method computationally analyses the...
Binding sites are often located in large pockets, and if their location on a protein’s surface is unknown, it can be predicted using various approaches. The energetic method computationally analyses the...
02:26Conservation of Protein Domains Over Different Proteins
Protein domains are small structurally independent units that are part of a single amino acid chain. Although these domains are often structurally independent, they may rely on synergistic effects to perform their functions as part of a larger protein. Protein domains may be conserved within the same organism, as well as across different organisms.
A limited set of protein domains often duplicate and recombine during evolution. These domains can be organized in different combinations to form...
A limited set of protein domains often duplicate and recombine during evolution. These domains can be organized in different combinations to form...
01:24Protein Organization
Proteins are polymers of amino acid residues. They are versatile and responsible for different cellular functions, including DNA replication, molecular transport, catalysis, and structural support. Proteins have a hierarchical structure comprising at least three levels of organization: primary, secondary, and tertiary structure. Some large proteins have a quaternary structure where individual protein subunits are linked together.
The primary structure of a protein is its amino acid sequence.
The primary structure of a protein is its amino acid sequence.
02:47Protein Families
Protein families are groups of homologous proteins; that is, they have similarities in amino acid sequences and three-dimensional structures. Protein families usually occur because of gene duplication, where an additional copy of a gene is inserted into the genome of an organism. Mutations that change the amino acids but still allow the protein to be properly synthesized, will lead to new protein family members. If these new proteins contain similar amino acids in key locations, protein...