Assessing generative model coverage of protein structures with SHAPES.

Area of Science:

• Computational Biology
• Structural Bioinformatics
• Artificial Intelligence in Biology

Background:

• Generative models excel at sampling protein structures but exhibit bias towards idealized designs.
• This bias leads to the neglect of complex structural motifs (e.g., loops) essential for protein function.
• Existing models inadequately represent the full diversity of the protein structure space.

Purpose of the Study:

• To introduce SHAPES (structural and hierarchical assessment of proteins with embedding similarity) for evaluating protein structure generative models.
• To assess the coverage of the protein structure space by five state-of-the-art generative models.
• To identify limitations in current generative models concerning the sampling of diverse protein structures.

Main Methods:

• Utilized structural embeddings across multiple hierarchical levels, from local geometries to global architectures.
• Employed Fréchet protein distance (FPD) to quantitatively measure distributional coverage of protein structures.
• Analyzed the frequency of tertiary motifs (TERMs) to corroborate findings on model coverage.

Main Results:

• Revealed substantial undersampling of the observed protein structure space by leading generative models.
• Demonstrated distinct coverage behaviors across different models, noise scales, and temperatures.
• Confirmed model limitations through analysis of tertiary motif frequencies.

Conclusions:

• Current generative models for protein structures do not adequately cover the full spectrum of biological possibilities.
• Improved protein sequence design and structure prediction methods are necessary to guide the development of more comprehensive models.
• Enhanced models are crucial for advancing the field of protein engineering and design.