Assessing the generalization capabilities of TCR binding predictors via peptide distance analysis

Area of Science:

• Immunology
• Computational Biology
• Machine Learning

Background:

• T Cell Receptor (TCR) and peptide-bound Major Histocompatibility Complex (pMHC) interactions are central to immune responses and immunotherapy development.
• Current machine learning (ML) models excel at predicting TCR-pMHC binding on training data but struggle with generalization to novel peptides, limiting therapeutic applications.

Purpose of the Study:

• To evaluate how the distance between training and testing peptide distributions affects ML model performance in predicting TCR-pMHC binding.
• To introduce and utilize a novel Distance Split (DS) algorithm for a more robust assessment of model generalization capabilities.

Main Methods:

• Assessed the impact of sequence-based and 3D structure-based distance metrics on ML model generalization.
• Employed state-of-the-art models including Attentive Variational Information Bottleneck (AVIB), NetTCR-2.0/2.2, and ERGO II variants.
• Introduced the Distance Split (DS) algorithm to control peptide distribution distances in training and testing sets.

Main Results:

• Lower 3D shape similarity between training and test peptides correlated with a more challenging out-of-distribution task, indicating poorer generalization.
• Sequence-based similarity showed an opposite trend, suggesting different implications for generalization.
• The findings underscore the utility of distance-based splitting for benchmarking TCR-pMHC binding predictors.

Conclusions:

• A distance-based splitting approach, particularly considering 3D structural similarity, is crucial for accurately evaluating the generalization of TCR-pMHC binding prediction models.
• This method can inform confidence scores for predictions on novel peptides based on their dissimilarity to training data.
• Integrating 3D structural information alongside sequence data may enhance the predictive accuracy of TCR-pMHC binding models.