Training data composition affects performance of protein structure analysis algorithms.

Area of Science:

• Structural biology
• Computational biology
• Biochemistry

Background:

• Accurate protein three-dimensional structures are vital for understanding molecular mechanisms and interactions.
• Machine learning models for protein structure prediction are crucial for protein engineering and drug development.
• Training data quality significantly impacts the performance of these machine learning models.

Purpose of the Study:

• To evaluate the impact of different experimental methods (X-ray crystallography, NMR, cryo-EM) on machine learning model training data bias.
• To assess how this bias affects model performance in tasks like accuracy estimation, protein sequence design, and catalytic residue prediction.
• To determine optimal strategies for composing training datasets to mitigate bias and enhance model generalizability.

Main Methods:

• Trained machine learning models on datasets comprising either all three structure types (X-ray, NMR, cryo-EM) or only X-ray data.
• Evaluated model performance on test sets derived from each experimental method.
• Analyzed the relationship between model performance and the biophysical properties of each structure determination method.

Main Results:

• Models trained solely on X-ray data performed worse on NMR and cryo-EM test sets.
• Including NMR and cryo-EM structures in training datasets mitigated performance disparities across structure types.
• Training on all three structure types did not degrade, and sometimes improved, performance on X-ray test sets.

Conclusions:

• The choice of experimental methods for protein structure determination introduces bias into machine learning training data.
• Diverse training datasets incorporating X-ray, NMR, and cryo-EM data enhance model robustness and generalizability.
• Consideration of the biophysical properties and the specific biochemical task is recommended when curating training sets for protein structure prediction models.