PRIME: An evaluation framework for protein representation inference and generalization in viral mutation space

Area of Science:

• Computational Biology
• Virology
• Machine Learning

Background:

• Protein language models (PLMs) show promise for protein fitness prediction.
• Extreme sequence homology in viral pathogens causes data leakage in standard validation, inflating performance metrics.
• Current PLM applications in viral pathogen surveillance lack real-world utility due to unreliable predictions.

Purpose of the Study:

• To develop a robust framework for evaluating viral threats using PLMs.
• To address the challenge of data leakage and improve the generalizability of PLMs for rapidly evolving viruses.
• To establish a new benchmark for applying PLMs in pathogen surveillance.

Main Methods:

• Introduced Protein Representation Inference for Mutation Evaluation (PRIME) framework.
• Integrated domain-specific fine-tuning with position-stratified validation.
• Utilized a dataset of 347,432 SARS-CoV-2 receptor binding domain (RBD) sequences for benchmarking.

Main Results:

• Position-stratified validation revealed random splits yield deceptive R² values (>0.90) for SARS-CoV-2 RBD sequences.
• Domain-specific fine-tuning of ESM-C 600M with stratified data achieved R² ~0.23 for predicting binding affinity and expression at unseen mutational sites.
• PRIME identified 3.03% of bat coronavirus sequences as candidates for experimental prioritization based on functional similarity.

Conclusions:

• PRIME sets a new benchmark for PLM application in pathogen surveillance.
• State-of-the-art models and fine-tuning, combined with stratified validation, offer biologically meaningful insights.
• This approach enhances understanding of pathogen evolution and zoonotic risk.