Observation selection bias in contact prediction and its implications for structural bioinformatics

Area of Science:

• Structural bioinformatics
• Computational biology
• Protein structure prediction

Background:

• Next Generation Sequencing rapidly expands protein sequence data, outpacing experimental structure determination.
• Current structural bioinformatics methods predict protein characteristics using evolutionary information from homologous sequences.
• Existing validation practices may inflate performance metrics due to data biases.

Purpose of the Study:

• To identify and quantify observational selection bias in structural bioinformatics.
• To demonstrate the impact of this bias on the performance of protein contact prediction methods.
• To provide a novel, bias-free dataset for robust method validation.

Main Methods:

• Analysis of sequence homology in protein structure databases (PDB) versus general sequence databases (Uniprot).
• Re-evaluation of two protein contact prediction methods using realistic evolutionary information.
• Development and release of the NOUMENON dataset for unbiased validation.

Main Results:

• A significant observational selection bias exists, favoring proteins with known structures in databases.
• Performance of contact prediction methods can decrease by up to 60% when accounting for this bias.
• The NOUMENON dataset provides a more accurate benchmark for evaluating prediction algorithms.

Conclusions:

• Current validation strategies for structural bioinformatics methods are likely flawed due to data bias.
• Accurate protein structure prediction requires addressing and correcting for observational selection bias.
• The NOUMENON dataset is crucial for the development of reliable protein contact prediction tools.