Assessing Generative Model Coverage of Protein Structures with SHAPES.

Area of Science:

• Computational Biology
• Structural Biology
• Machine Learning

Background:

• Generative models efficiently sample protein structures but may introduce bias.
• This bias favors idealized structures, neglecting crucial loops and complex motifs essential for protein function.

Purpose of the Study:

• To evaluate five state-of-the-art generative models for protein structures.
• To assess their coverage of the true protein structure space using a novel evaluation method.

Main Methods:

• Introduction of SHAPES (Structural and Hierarchical Assessment of Proteins with Embedding Similarity).
• Utilizing structural embeddings across multiple hierarchies (local to global).
• Quantifying distributional coverage using Fréchet Protein Distance (FPD) and analyzing TERtiary Motifs (TERMs).

Main Results:

• State-of-the-art generative models substantially undersample the observed protein structure space.
• Model performance varies with sampling noise scales and temperatures.
• Analysis of TERtiary Motifs (TERMs) corroborates findings on structural coverage.

Conclusions:

• Current generative models exhibit limitations in covering the full spectrum of protein structures.
• Improved sequence design and structure prediction methods are vital for developing models with broader coverage.
• Further research is needed to enhance generative models for more comprehensive protein structure representation.